View · Search · Index
No registered users in community xowiki
in last 10 minutes

[Xotcl] Announcement: Next Scripting Framework 2.1.0 available

From: Gustaf Neumann <neumann_at_wu.ac.at>
Date: Wed, 28 Dec 2016 13:20:14 +0100

Dear Community,

We are pleased to announce the availability of the Next Scripting
Framework (NSF) 2.1.0 as a Christmas present to the community.

Over the last years, NSF is constantly being developed based
on experience in largish projects. To document these changes
we have decided it to bump the version numbers and make
a new release.

Since the release of NSF 2.0.0, there have been more than 450
commits to our code repository. For this release NSF was
tested in 192 build configurations with Tcl 8.5.19 and
Tcl 8.6.6 on Linux and Mac OS X (both using recent gcc and
clang). Common build configurations have been tested under Windows
(Windows 2012 Server: VS15 & MSVC 19, MinGW).

Diff stats since 2.0.0:
  186 files changed, 26475 insertions(+), 15671 deletions(-)

Major changes relative to 2.0.0 are:

- Improved debugging/tracing/profiling support (NSF)
- Deprecated and debug modes for NSF procs and methods (NSF, NX, XOTcl2)
- Script-level access to NSF parameter/argument parser (NSF).
- Slot-trace reform (NX, XOTcl2)
- Updated MongoDB interface to the newest mongo-c-driver (NSF, NX)
- Improved scripted shells (e.g., nxsh, xotclsh)

Below is a summary of are the changes in more details. The detailed
changelog is available at
https://next-scripting.org/xowiki/download/file/ChangeLog-2.0.0-2.1.0.log

NSF 2.1.0 (containing NX 2.1.0 and XOTcl 2.1.0) can be obtained from
https://next-scripting.org/. Please report issues and wishes by opening
a ticket at https://sourceforge.net/p/next-scripting/tickets/.

Best regards
- Gustaf Neumann
- Stefan Sobernig

========================================================================
New Features

- NSF:

   * Improved debugging/tracing/profiling support:

     + NSF procs and methods can be marked "deprecated" and/or
       "debug-enabled". For this, all NSF proc and method definitions
       in NX and XOTcl2 now accept the new flags "-debug" and
       "-deprecated". The low-level interface (nsf::method::property)
       enables introspection and selective enabling during runtime.

           nsf::proc ... ?-debug? ?-deprecated? ... /name/ /argspec/ /body/
           nsf::method::property /obj/ /method/ deprecated ?0|1?
           nsf::method::property /obj/ /method/ debug ?0|1?

       A debug-enabled NSF proc or method will be reported on entering
       and exiting the proc and method (similar to a traced proc/
       method incl. a time probe; see above):

           % nsf::proc -debug foo {} {return 123}
                 % foo
           Debug: call(1) - foo
           Debug: exit(1) - ::nsf::procs::foo 72 usec -> 123

       The enter and exit messages can be tailored and redirected
       at the script level by redefining "::nsf::debug::call" and
       "::nsf::debug::exit", respectively.

       To actively manage API evolution, NSF procs and methods can be
       marked "deprecated". Using deprecated procs/ methods will result
       in a warning:

           % nsf::proc -deprecated foo {} {;}
           % foo
           Warning: *** proc foo is deprecated.

       The warning message can be tailored and redirected
       at the script level by redefining the proc "::nsf::deprecated".

     + NSF procs and methods can be profiled (and optionally traced)
       when profiling support is configured during compilation

          ./configure --enable-profile (default: disabled)

       This can be controlled via

          nsf::__profile_trace -enable /bool/ ?-verbose /bool/?
          nsf::__profile_get
          nsf::__profile_clear

       When profiling is enabled, the profiling data is kept in memory
       and can be obtained via "nsf::__profile_get" in form of a list
       structure containing the wall clock time since start of
       profiling, the aggregated ms and lists containing object times
       (what time was spent in which object/classes), method times
       (what time was spent in which methods) and proc data (what time
       was used in which procs) followed by trace data (showing calls
       and exits of methods/procs/cmds). When "nsf::__profile_trace" is
       called with the "-verbose" flag, the trace is printed via
       "nsf::log".

   * Add script-level access to the NSF argument parser to conveniently
     process, e.g., non-positional parameters/arguments and value
     checkers. See
     https://next-scripting.org/xowiki/docs/nx/tutorial/index1#_parameters
     for the details.

     The command

          ::nsf::parseargs /paramspec/ /arglist/

     can be used to parse arglist based on the paramspec to set the
     parsed arguments in the local context. For example, the command

              % nsf::parseargs {{-x:integer 1} y} {123}
         % set x
         1
         % set y
         123

      will define variables x holding "1" and y holding "123" for the
      current Tcl scope.

   * Minor logging reform (NsfLog, interp): "nsf::configure debugLevel
     /severity/" will print error messages at a level equal or greater
     than the given severity (0, 1, 2, 3). For example, "nsf::configure
     debugLevel 0" will print any "nsf::log" message, and
     "nsf::configure debugLevel 3" will print just "nsf::log" level 3
     (error) messages while omitting warnings etc. This does not entail
     changed logging semantics, but helped remove some confusion at the
     NSF/C level.

   * Improved NSF/C code generator (gentclAPI.tcl): Allows for
     specifying and generating enum types.

   * Misc:

     + New flag "-notrace" for "nsf::var::set" and "nsf::var::get" to
       implement ability to use these low-level commands for accessing
       variables without firing Tcl traces.

     + New cmd "nsf::method::forward::property" for reading for writing
       introspection of forwarders. This is important for the
       serializer to map the per-object forwarder when different target
       objects are specified.

     + New option for call-stack introspection: "nsf::current
       level". It returns the stack level of the currently executing
       NSF method (like "info level"), or an empty string outside of an
       NSF context.

- NX:

   * NX method definitions ("alias", "forward", and "method") now
     accept the optional flags "-deprecated" and "-debug". See NSF
     section above for usage details.

        /cls/ public alias ?-deprecated? ?-debug? /method/ ...
        /cls/ public forward ?-deprecated? ?-debug? /method/ ...
        /cls/ public method ?-deprecated? ?-debug? /method/ ...
        /obj/ public object alias ?-deprecated? ?-debug? /method/ ...
        /obj/ public object forward ?-deprecated? ?-debug? /method/ ...
        /obj/ public object method ?-deprecated? ?-debug? /method/ ...

   * Improved object and method introspection: Newly added
     introspection subcommands complete the set of introspection
     commands and reflect the newly introduced method options.

        /cls/ info method callprotection
        /cls/ info method debug
        /cls/ info method deprecated
        /obj/ info baseclass
        /obj/ info object method callprotection
        /obj/ info object method debug
        /obj/ info object method deprecated

   * Reform of traces on slot-managed object variables: Revised the
     interface and semantics for the interaction of variable traces and
     slots. It is now possible to specify in the definition of a
     "property" or "variable" whether the slot-accessor methods
     ("value=get", "value=set") should be fired whenever a variable is
     read/written or when it is initialized to a default value for the
     first time ("value=default").

        /obj/ object property|variable ?-trace set|get|default? ...
        /cls/ property|variable ?-trace set|get|default? ...
        
     See the nx::Object and nx::Class man pages for details.
     This supersedes the experimental interface available in
     XOTcl 1.* based on "initcmd", "valuecmd" and "valuechangedcmd".

   * Documentation (API and examples):

     + Added

       Rosetta implementations: Inheritance/ single, add object
       variable dynamically, tree traversal, and tokenizer.
        
     + Updated

       Object.man: Added missing description on "info lookup
       parameters" and "info lookup syntax"; corrected description of
       "copy"; added "info baseclass"

       {alias|forward|method}.man.inc: Added documentation of
       "-debug" and "-deprecated" switches.

       current.man: Added description of 'level' option.

       Rosetta implementations: Inheritance/multiple, polymorphic
       copy, multiple distinct objects

- XOTcl2:

   * XOTcl2 method definitions ("proc", "instproc", "forward", and
     "instforward") now accept the optional flags "-deprecated" and
     "-debug". See NSF section above for usage details.

         /cls/ instforward -deprecated|-debug /method/ ...
         /cls/ instproc -deprecated|-debug /method/ ...

         /obj/ forward -deprecated|-debug /method/ ...
         /obj/ proc -deprecated|-debug /method/ ...

   * New -return flag: XOTcl2 method definitions can now specify a
     return-value checker, similar to NX methods.

     /cls/ instproc -returns /method/ ...
     /obj/ proc -returns /method/ ...


- Shells:

   * There is now an improved and packaged shell implementation:
     nx::shell2. It is used by all four shell scripts. nx::shell2
     builds on Tcl's event loop, rather than while + update. This
     avoids blocking the Tk main window (update). In addition,
     nx::shell2's behavior is more akin to Tcl's native shells (e.g.,
     no extra lines on enter, catches EOF). The Tcl package can be
     conveniently sourced, e.g., in Tclkit main scripts.

   * The new shells accept script input now via stdin or as provided
     via "-c" in the command line.

   * They are more robust: Don't quit due to inner [return] calls,
      [exit] is handled gracefully.

   * Added tests for the shells to the regression test suite
     (shells.test).


- Traits:

   * The namespace "nx::traits::XXX" was renamed to "nx::trait::XXX" in
     order to improve naming orthogonality with the trait management
     package "nx::trait".

- MongoDB interface upgrade:

   * Upgraded MongoDB and drivers to mongodb-c-driver 1.5.1, libbson
     1.5.1 and MongoDB v3.4.0.

   * The upstream mongodb-c-driver has deprecated the old OP_QUERY syntax
     with $-modifiers and old names like $orderby and $query, which will
     be removed from the driver. The new interface is more in the mongodb
     style, many former parameters (e.g. -limit, -skip, -project, -sort)
     went to the new "-opts" argument.

       mongo::collection::query /collection/ /filter/ ?-opts /opts/?
       mongo::cursor::find /collection/ /filter/ ?-opts /opts/?

     See the begin of nsf-mongo.test file for examples for queries with
     the old and new interface. The high-level interface for nx::Object
     is unchanged.

   * Support for new binary type "decimal128"
   
- Maintenance & bug fixes:

   * Tcl command resolvers and command literals:

     Between Tcl 8.6.0 and (including) Tcl 8.6.6, there was a problem
     in the interaction between Tcl 8.6 and the command resolvers of
     NSF in certain corner cases and Tcl's command literals. The issue
     and patches were reported upstream to the Tcl core team, was
     recognized and will enter an upcoming patch release (8.6.7). Our
     regression test-suite was extended accordingly to track the issue.

     See Tcl Fossil tickets d4e7780ca1 and 3418547.

   * Finalizing NSF and NSF object systems:

     In this release, the finalization procedures of NSF (on process
     exits, thread exits, and interp teardowns) has been revised. For
     example, NSF's ExitHandler is guaranteed to be called just once in
     the above scenarios (e.g., to avoid double frees of NSF
     structures).

   * Fixed a long-standing and not easy reproducible bug when deleting
     classes in multiple nested diamond inheritance structures.

   * UnsetTracedVars:

     Provide for a two-pass deletion logic during object shutdown, to
     account for unset traces possibly reviving an object variable
     under deletion. This corresponds to a recent memory leak fix to
     Tcl itself. See also Tcl Fossil ticket 4dbdd9af144dbdd9af14.

   * Serializer:

     The combined NX and XOTcl2 serializer has been extended to cover
     newly added features (esp., NSF procs, method flags) and has been
     improved based on experiences from multi-threaded NSF/Tcl
     applications (NaviServer). Improvements and fixes relate to "info"
     methods and forwarders.

   * Tcl 8.7:

     Preliminary support for a future Tcl 8.7 (as of Dec 2, 2016;
     commit 71fa1f9c91). NSF compiles under 8.7a0 and its regression
     tests execute successfully.

   * Misc (esp. NSF/C):

     + Refactoring: There is now a common infrastructure for
       hash-tables using function pointers as keys (as required for
       NSF/C command definitions and enumerations).
     + Removed all implicit type-conversions, signed/unsigned
       comparisons as flagged by the following gcc/clang flags (ensure
       cleanness on 64bit) -Wconversion -Wsign-conversion
       -Wfloat-conversion -Wsign-compare.
     + Reduced variable scopes.
     + Guarded against potential Tcl_Obj leaks, esp. when using
       "NsfMethodNamePath", as indicated by valgrind.
     + Refactored functions for topological sorting ("TopoSort") to
       avoid code redundancies.
     + Addressed more than 18 code-quality issues as indicated by
       Coverity Scan, incl. control-flow issues (risking NULL
       dereferencing), dead code branches, etc.
     + Removed uses of deprecated functions, e.g. "Tcl_AppendResult".
     + Based on the improve NSF/C code generator (see above), proper
       enum types are used in generated function signatures and the
       respective control structures (switch).

   * Extended regression test suite:
     The test suite contains (for Tcl 8.6.6) now 5852 tests.

- Packaging & distribution:

   * Updated TEA to 3.10
   * Support for MSVC 1900 (VS 2015; win/makefile.vc)
   * Debian package: https://packages.qa.debian.org/n/nsf.html
   * MacPorts port: https://trac.macports.org/browser/trunk/dports/lang/nsf/Portfile
   * Part of the KitCreator battery: https://kitcreator.rkeene.org/fossil/
   * Part of the kbskit battery: https://sourceforge.net/projects/kbskit/