Dear Community, We are pleased to announce the availability of the Next Scripting Framework (NSF) 2.1.0. This is a maintenance release. Since the release of 2.0.0, there have been more than 450 commits to our code repository. The Next Scripting Framework was tested in 192 build configurations with Tcl 8.5.19 and Tcl 8.6.6 on Linux and Mac OS X (both using recent gcc and clang). Common build configurations have been tested under Windows (Windows 2012 Server: VS15 & MSVC 19, MinGW). Diff stats since 2.0.0: 185 files changed, 33023 insertions(+), 17848 deletions(-) Major changes relative to 2.0.0 are: - Improved debugging/tracing/profiling support (NSF) - Deprecated and debug modes for NSF procs and methods (NSF, NX, XOTcl2) - Script-level access to NSF parameter/argument parser (NSF). - Slot-trace reform (NX, XOTcl2) - Updated MongoDB interface to the newest C-driver - Improved scripted shells (e.g., nxsh, xotclsh) New Features - NSF: * Improved debugging/tracing/profiling support: + NSF procs and methods can be marked "deprecated" and/or "debug-enabled". For this, all NSF proc and method definitions in NX and XOTcl2 now accept the new flags "-debug" and "-deprecated". The low-level interface (nsf::method::property) enables introspection and selective enabling during runtime. nsf::proc ... ?-debug? ?-deprecated? ... /name/ /argspec/ /body/ nsf::method::property /obj/ /method/ deprecated ?0|1? nsf::method::property /obj/ /method/ debug ?0|1? A debug-enabled NSF proc or method will be reported on entering and exiting the proc and method (similar to a traced proc/ method incl. a time probe; see above): % nsf::proc -debug foo {} {return 123} % foo Debug: call(1) - foo Debug: exit(1) - ::nsf::procs::foo 72 usec -> 123 The enter and exit messages can be indirected at the script level by (re-)defining "::nsf::debug::call" and "::nsf::debug::exit", respectively. To actively manage API evolution, NSF procs and methods can be marked "deprecated". Using deprecated procs/ methods will result in a warning: % nsf::proc -deprecated foo {} {;} % foo Warning: *** proc foo is deprecated. The warning can be indirected at the script level by (re-)defining a proc "::nsf::deprecated". + NSF procs and methods can be profiled (and optionally traced) when profiling support is configured during compilation ./configure --enable-profile (default: disabled) This can be controlled via nsf::__profile_trace -enable /bool/ ?-verbose /bool/? nsf::__profile_get nsf::__profile_clear When profiling is enabled, the profiling data is kept in memory and can be obtained via "nsf::__profile_get" in form of a list structure containing the wall clock time since start of profiling, the aggregated ms and lists containing object times (what time was spent in which object/classes), method times (what time was spent in which methods) and proc data (what time was used in which procs) followed by trace data (showing calls and exits of methods/procs/cmds). When "nsf::__profile_trace" is called with the "-verbose" flag, the trace is printed via "nsf::log". * Add script-level access to the NSF argument parser to conveniently process, e.g., non-positional parameters/arguments and value checkers. See https://next-scripting.org/xowiki/docs/nx/tutorial/index1#_parameters for the details. The command ::nsf::parseargs /paramspec/ /arglist/ can be used to parse arglist based on the paramspec to set the parsed arguments in the local context. For example, the command: % nsf::parseargs { { -x:integer 1} y } {123} % set x 1 % set y 123 will define variables x holding "1" and y holding "123" for the current Tcl scope. * Minor logging reform (NsfLog, interp): "nsf::configure debugLevel /severity/" will print error messages at a level equal or greater than the given severity (0, 1, 2, 3). For example, "nsf::configure debugLevel 0" will print any "nsf::log" message, and "nsf::configure debugLevel 3" will print just "nsf::log" level 3 (error) messages while omitting warnings etc. This does not entail changed logging semantics, but helped remove some confusion at the NSF/C level. * Improved NSF/C code generator (gentclAPI.tcl): Allows for specifying and generating enum types. * Misc: + New flag "-notrace" for "nsf::var::set" and "nsf::var::get" to implement ability to use these low-level commands for accessing variables without firing Tcl traces. + New cmd "nsf::method::forward::property" for reading for writing introspection of forwarders. This is important for the serializer to map the per-object forwarder when different target objects are specified. + New option for call-stack introspection: "nsf::current level". It returns the stack level of the currently executing NSF method (like "info level"), or an empty string outside of an NSF context. - NX: * NX method definitions ("alias", "forward", and "method") now accept the optional flags "-deprecated" and "-debug". See NSF section above for usage details. /cls/ public alias ?-deprecated? ?-debug? /method/ ... /cls/ public forward ?-deprecated? ?-debug? /method/ ... /cls/ public method ?-deprecated? ?-debug? /method/ ... /obj/ public object alias ?-deprecated? ?-debug? /method/ ... /obj/ public object forward ?-deprecated? ?-debug? /method/ ... /obj/ public object method ?-deprecated? ?-debug? /method/ ... * Improved object and method introspection: Newly added introspection subcommands complete the set of introspection commands and reflect the newly introduced method options. /cls/ info method callprotection /cls/ info method debug /cls/ info method deprecated /obj/ info baseclass /obj/ info object method callprotection /obj/ info object method debug /obj/ info object method deprecated * Reform of traces on slot-managed object variables: Revised the interface and semantics for the interaction of variable traces and slots. It is now possible to specify in the definition of a "property" or "variable" whether the slot-accessor methods ("value=get", "value=set") should be fired whenever a variable is read/written or when it is initialized to a default value for the first time ("value=default"). /obj/ object property|variable ?-trace set|get|default? ... /cls/ property|variable ?-trace set|get|default? ... See the Object and Class manpages for details. This supersedes the experimental interface available in XOTcl: "initcmd", "valuecmd" and "valuechangedcmd". * Documentation (API and examples): + Added Rosetta implementations: Inheritance/ single, add object variable dynamically, tree traversal, and tokenizer. + Updated Object.man: Added missing description on "info lookup parameters" and "info lookup syntax"; corrected description of "copy"; added "info baseclass" {alias|forward|method}.man.inc: Added documentation of "-debug" and "-deprecated" switches. current.man: Added description of 'level' option. Rosetta implementations: Inheritance/multiple, polymorphic copy, multiple distinct objects - XOTcl2: * XOTcl2 method definitions ("proc", "instproc", "forward", and "instforward") now accept the optional flags "-deprecated" and "-debug". See NSF section above for usage details. /cls/ instforward -deprecated|-debug /method/ ... /cls/ instproc -deprecated|-debug /method/ ... /obj/ forward -deprecated|-debug /method/ ... /obj/ proc -deprecated|-debug /method/ ... * New -return flag: XOTcl2 method definitions can now specify a return-value checker, similar to NX methods. /cls/ instproc -returns /method/ ... /obj/ proc -returns /method/ ... - Shells: * There is now an improved and packaged shell implementation: nx::shell. It is used by all four shell scripts. nx::shell builds on Tcl's event loop, rather than while + update. This avoids blocking the Tk main window (update). In addition, nx::shell's behavior is more akin to Tcl's native shells (e.g., no extra lines on enter, catches EOF). The Tcl package can be conveniently sourced, e.g., in Tclkit main scripts. * The new shells accept script input now via stdin or as provided via "-c" in the command line. * They are more robust: Don't quit due to inner [return] calls, [exit] is handled gracefully. * Added tests for the shells to the regression test suite (shells.test). - Traits: * The namespace "nx::traits::XXX" was renamed to "nx::trait::XXX" in order to improve naming orthogonality with the trait management package "nx::trait". - MongoDB interface upgrade: * Upgraded MongoDB and drivers to mongodb-c-driver 1.5.1, libbson 1.5.1 and MongoDB v3.4.0. * The upstream mongodb-c-driver has deprecated the old OP_QUERY syntax with $-modifiers and old names like $orderby and $query, which will be removed from the driver. The new interface is more in the mongodb style, many former parameters (e.g. -limit, -skip, -project, -sort) went to the new "-opts" argument. mongo::collection::query /collection/ /filter/ ?-opts /opts/? mongo::cursor::find /collection/ /filter/ ?-opts /opts/? See the begin of nsf-mongo.test file for examples for queries with the old and new interface. The high-level interface for nx::Object is unchanged. * Support for new binary type "decimal128" - Maintenance & bug fixes: * Tcl command resolvers and command literals: Between Tcl 8.6.0 and (including) Tcl 8.6.6, there was a problem in the interaction between Tcl 8.6 and the command resolvers of NSF in certain corner cases and Tcl's command literals. The issue and patches were reported upstream to the Tcl core team, was recognized and will enter an upcoming patch release (8.6.7). Our regression test-suite was extended accordingly to track the issue. See Tcl Fossil tickets d4e7780ca1 and 3418547. * Finalizing NSF and NSF object systems: In this release, the finalization procedures of NSF (on process exits, thread exits, and interp teardowns) has been revised. For example, NSF's ExitHandler is guaranteed to be called just once in the above scenarios (e.g., to avoid double frees of NSF structures). * Fixed a long-standing and not easy reproducible bug when deleting classes in multiple nested diamond inheritance structures. * UnsetTracedVars: Provide for a two-pass deletion logic during object shutdown, to account for unset traces possibly reviving an object variable under deletion. This corresponds to a recent memory leak fix to Tcl itself. See also Tcl Fossil ticket 4dbdd9af144dbdd9af14. * Serializer: The combined NX and XOTcl2 serializer has been extended to cover newly added features (esp., NSF procs, method flags) and has been improved based on experiences from multi-threaded NSF/Tcl applications (NaviServer). Improvements and fixes relate to "info" methods and forwarders. * Tcl 8.7: Preliminary support for a future Tcl 8.7 (as of Dec 2, 2016; commit 71fa1f9c91). NSF compiles under 8.7a0 and its regression tests execute successfully. * Misc (esp. NSF/C): + Refactoring: There is now a common infrastructure for hash-tables using function pointers as keys (as required for NSF/C command definitions and enumerations). + Removed all implicit type-conversions, signed/unsigned comparisons as flagged by the following gcc/clang flags (ensure cleanness on 64bit) -Wconversion -Wsign-conversion -Wfloat-conversion -Wsign-compare. + Reduced variable scopes. + Guarded against potential Tcl_Obj leaks, esp. when using "NsfMethodNamePath", as indicated by valgrind. + Refactored functions for topological sorting ("TopoSort") to avoid code redundancies. + Addressed more than 18 code-quality issues as indicated by Coverity Scan, incl. control-flow issues (risking NULL dereferencing), dead code branches, etc. + Removed uses of deprecated functions, e.g. "Tcl_AppendResult". + Based on the improve NSF/C code generator (see above), proper enum types are used in generated function signatures and the respective control structures (switch). * Extended regression test suite: The test suite contains (for Tcl 8.6.6) now 5852 tests. - Packaging & distribution: * Updated TEA to 3.10 * Support for MSVC 1900 (VS 2015; win/makefile.vc) * Debian package: https://packages.qa.debian.org/n/nsf.html * MacPorts port: https://trac.macports.org/browser/trunk/dports/lang/nsf/Portfile * Part of the KitCreator battery: https://kitcreator.rkeene.org/fossil/ * Part of the kbskit battery: https://sourceforge.net/projects/kbskit/ The detailed changelog is available at https://next-scripting.org/xowiki/download/file/ChangeLog-2.0.0-2.1.0.log  The Next Scripting Framework 2.1.0 (containing NX 2.1.0 and XOTcl 2.1.0) can be obtained from https://next-scripting.org/ . Please report issues and wishes by opening a ticket at https://sourceforge.net/p/next-scripting/tickets/ . Best regards - Gustaf Neumann - Stefan Sobernig