Standard ML of New Jersey Change Log

Changed the layout of the SML stack frame on the AMD64 architecture to make it compatible with the way that LLVM spills registers. Essentially, this just involved swapping the order of the swap area and the ML stuff. We took this opportunity, however, to localize up the representation of this information in the compiler.

John Reppy

Changed the format of the "magic string" in the header of binfiles. The new format is "arch-version", where the architecture name is limited to at most seven bytes and the version is limited to at most eight bytes. The string is padded with spaces to a total length of 16 bytes.

John Reppy

Various pretty-printer bug fixes:

John Reppy

Fixed bug #254 (Real.fromLargeInt produces negative results). The problem was because the digit size for IntInf.int is only one bit smaller than the default int and the scaling factor rbase was being computed using the InlineT.Real64.from_int function (so rbase ends up being negative). Thus it would return incorrect results whenever the IntInf representation involved more than one digit. This is a bug on both 32-bit and 64-bit systems. The fix was to switch to using InlineT.Real64.from_int{32,64} to convert rbase and digits to real values.

John Reppy

Fixed bug #267 (Returns an incorrect result for a calculation on Position.int for 32-bit mode). The problem was that on 32-bit machines, 64-bit division is implemented by the IntInf module with the result then being converted to 64-bits. The conversion used did not test for overflow in the result.

John Reppy

Changed the semantics of the spans returned by ml-ulex so that the second component of a span is the position of the rightmost character in the token (instead of the character following the token). Specifically, the span \((p_1, p_2)\) specifies the \(p_2 - p_1 + 1\) characters that start with the character at position \(p_1\) and run to \(p_2\) (inclusive). This change avoids a potential problem when the span of a token ends at the last character in a file (when the input is spread across multiple files).

John Reppy

Simplified the binfile representation by removing the option of having multiple code objects. Many years ago, we would split the code for a compilation unit into multiple independent functions so that the garbage collector could reclaim code that was only executed once (or was not referenced). The actual splitting of the code in the CpsSplitFun functor (CPS/clos/cps-split.sml) was replaced by a dummy implementation at some point, so we have not been generating multiple code objects for some time. Therefore, we have simplified the code generator to assume only one code object and have changed the binfile import/export code to only support a single code object per binfile.

Also made this change to the bootstrap loader (kernel/boot.c).

John Reppy

Reverted some of the pretty printing changes that were made in 110.98 to the 110.97 version (the renaming of PrettyPrintNew to PrettyPrint and the directory reorganizations were unchanged). These changes fix bugs #266 (Pretty printing regression in SML/NJ 110.98), #268 (Polymorphic Type Pretty Printing Regression), and #271 (pretty printer regression for structure binding).

John Reppy

Fixed bug #269 (Word64.fromString causes an Overflow for greater than 2³²-1). This bug was the result of constants from the 32-bit version of the code not getting updated for the 64-bit version. Scanning of both hexadecimal and octal representations of both integers and words were affected.

John Reppy

Added an additional lowering pass for the STREQL primop. This primop is generated to implement pattern matching against string literals. Previously it was unrolled in the MLRISC code generator, but we now do the unrolling in CPS. The reason for this change is that implementing the unrolling in the LLVM code generator would be complicated because of the need to introduce phi nodes in one of the branches.

The unrolling in CPS is somewhat different from before in that we now bake the literal string being tested into the equality tests.

Also changed the representation of the primop to include the string being tested against and removed the STRNEQ primop.

John Reppy

Implemented Basis Library proposal 2020-001 (Addition of Universal module).

John Reppy

Changed the config/install.sh script so that the default size is 64 bits for any machine that reports its machine as "x86_64."

John Reppy

Fixed bug #260 (Perform divide on Position.int crashes with FPE on Linux). The fix required adding SIGFPE as a second source of Overflow exceptions on Linux/amd64 machines.

John Reppy

Fixes for structure and signature pretty printing problems that were introduced in the extensive pretty printer/pretty printing modifications around revision 6291.

Files affected include ElabData/modules/ppmod.sml, ElabData/types/pptype.sml, TopLevel/print/ppdec.sml, and Basics/print/pputil.{sig,sml}.

The pretty printing for modules still seems quite fragile, so there are likely to be more pretty printing problems to be fixed later. In particular, pretty printing of functor and functor signature declarations haven’t been tested.

Dave MacQueen

Added a new lowering pass following CPS optimization, but before closure conversion. This pass includes the previous passes for 64-bit operations on 32-bit matchines and for conversions involving IntInf.int. It also adds lowering for div and mod to native machine division (i.e., quot and rem) and for trapping conversions.

The purpose of this change is to simplify code generation in preparation for migrating to a LLVM-based backend.

John Reppy

Fix for bug #261 (Weird "calc_strictness" message being printed). Rewrote the function ElabUtil.calc_strictness and moved it to TypesUtil.calcStrictness.

Dave MacQueen

The HTMLDev structure in the pretty-printing library has been moved into its own library (pp-extras-lib.cm) and renamed as HTML3Dev. This change removes the dependency of pp-lib.cm on html-lib.cm, which allowed us to remove all mention of html-lib.cm from the compiler CM files.

John Reppy

Completed the removal of the trigonometry operators from the primops. This change also allowed the removal of extension support from the MLRISC code generator for the x86.

John Reppy

Disabled the use of the hardware instructions for the basic trig functions on the x86. Doing so simplifies cross compilation from non x86 hosts and also paves the way to removing the operations from the compiler’s primitive operators.

John Reppy

Improved the CPS contraction phase by adding strength-reduction optimizations to ContractPrim. These include recognizing when multiplications and divisions by powers of 2 can be replaced by shifts. Previously, these sorts of optimizations were provided by MLRISC, but we plan to simplify the CPS IR prior to code generation by replacing div and mod operations with native machine arithmetic, which would prevent MLRISC from making the optimizations.

John Reppy

The MLRISC instruction selector for the x86 and amd64 targets erroneously assumed that the idiv instruction sets the OF (overflow) condition code when dividing the largest negative number by ~1. In fact, such a division operation traps, which is okay, because the runtime system maps the trap to the Overflow exception. Since the check for overflow is unnecessary, it has been removed from the files MLRISC/amd64/mltree/amd64-gen.sml and MLRISC/x86/mltree/x86.sml.

Note that MLRISC's non-trapping signed division operations can actually trap on overflow, but this was true before this change.

John Reppy

Changed the semantics of the --debug command-line option for ml-antlr. Previously this option replaced the actions with a print expression, but that limited its usefulness because of type errors in the generated code. The new behavior is to preserve the existing actions and just add the printing code.

John Reppy

Added a pass to the elaborator that check for variables that are bound, but never referenced. This check can be controlled by the Control.Elab.unusedWarn flag. Unused top-level variables are not reported (unless they are bound in the local part of a local declaration).

The check is currently disabled because of false positives caused by a transformation in the type checker. For example, the following function declaration:

gets represented by the following Absyn:

where the instance of g bound to #2 tmp is unused. This transformation is done by the wrapRECdec function. The unused-variable implementation was influenced by Jacob Van Buren’s patch for Version 110.82.

John Reppy

The LambdaVar.lvar type is represented as an integer; since the earliest days of the compiler this representation has been concrete, which meant that the type system was not able to provide any guarantees that int`s and `lvar`s were not being mixed up. As of this change, `LambdaVar.lvar is now an abstract equality type (internally, it is still and int). Some comments about the changes:

John Reppy

Fixed bug #256 (Ref.exchange incorrect). The original source of the bug was the Basis Library sample code, which has also been fixed.

John Reppy

Changed the printing of tyvars; e.g., an OVLDV tyvar introduced by an occurrence of the overloaded operator "*" that also acquires the equality attribute will be printed as ''Z[OL(*)].

Dave MacQueen

Eliminated AbsDec and ABSdec constructors

Dave MacQueen

File structure reoganization: the top-level compiler/Semant directory was eliminated. Remaining relevant subdirectories were Semant/pickle, which moved to ElabData, and Semant/prim, whose two files, prim-env.sml and primop-bindings.sml moved to the existing ElabData/prim directory. CM files compiler/core.cm and ElabData/elabdata.cm changed accordingly.

Dave MacQueen

Fixed bug #220

Dave MacQueen

Fixed bug #214

Dave MacQueen

Fixed bug #209

Dave MacQueen

Some clean up in the ml-lex/lexgen.sml code. Replaced the one-off implementation of finite maps with the RedBlackMapFn functor from the SML/NJ Library. Also got rid of the uses of polymorphic equality by changing token equality tests to pattern matching.

John Reppy

Turned several functors (ElabModFn, ElabTopFn, SigMatchFn, etc. into structures and removed the redundant functor application files (and the directories) in Semant/elaborate and Semant/modules.

Dave MacQueen

Fixed bugs #195 and #196

Dave MacQueen

Various minor changes related to the heap2exec and heap2asm programs.

John Reppy

Addressed bug #247 (@SMLVersion should report 64/32 bit) by adding a new command-line option (@SMLwordsize) to the .run-sml command script. Specifying this option will cause the the wordsize to be printed (either 32 or 64) and then the program will exit.

John Reppy

Fix for bug #252 (Boyer Benchmark Compile Failure). This crash was caused by a typo in the CPS/main/build-literals.sml code that caused an incorrect opcode to be generated for SAVE/LOAD instructions when the offset was >= 256.

John Reppy

Fixed the calculation of the maximum array/vector length for 64-bit targets. We had been using the calculation for 32-bit targets.

John Reppy

Fix for bug #245 (Lazy data types result in Compiler Bug error). The problem was that a number of symbols (e.g., deref) had been dropped from the _Core structure, but were required to support the lazy (and profiling) features in the compiler. The symbols have been reinstated and a comment has been added to explain why they are being included.

John Reppy

Fix for bug #244 (Compiler bug: PPObj: ppFields in ppval.sml). The code generator was using the wrong length tag for raw64 records on 64-bit machines (twice the length).

John Reppy

Clean up various issues in the configuration/build machinery for asdl. This includes a fix for bug #240 (Non-default 64-bit installation build failure)

John Reppy

Fix bug #239 (Date.toTime is incorrect (by a factor of 10E9)). Thanks to Johannes 5 Joemann for both the report and fix.

John Reppy

Bug fix for a problem where ^C (and other signals might be ignored). The fix is to use word-sized fields in the VProc state vector so that the word-sized move operations in the assembly code do not clobber adjacent fields.

John Reppy

Fix for bug #234 (Converting NaN to a string causes an infinite loop on 64-bit machines). The problem was in MLRISC/amd64/mltree/amd64-gen.sml, which was not generating comparisons that work correctly when the arguments are unordered.

John Reppy

Removed assertion checking from the amd64 runtime makefiles. It has not turned up any errors since 110.94 was released, so we will assume that things are working the way that they should.

John Reppy

Bugfix for bug #237 (heap2exec script fails on 110.95). The fix was provided by Kirill Boltaev.

John Reppy

Changed the default installation size to 64 bits on macOS 10.14 Mojave and later.

John Reppy

Fixed some code rot in the eXene sources (bug #233). With the LargeWord module changing from Word32 to Word64, there were a few places were things broke.

John Reppy

Added support for 64-bit executables on FreeBSD. As part of this effort, we fixed a couple of regressions (makefile issues) for the 32-bit version on FreeBSD that were introduced when the X86.prim.asm file was rewritten. We also switch from BSD signal handling to POSIX signal handling, since that is what we use for most other systems.

John Reppy

Fixed config/install.sh script, which was not passing the size option to the .link-sml, which caused confusion for the "-64" flag. This problem was later reported as bugs #235 and #236.

John Reppy

Many years ago, SML/NJ had a bytecode interpreter, but it was mostly removed from the system a long time ago. There were, however, some remnants of it in the runtime system. These have now been removed.

Having made this change, the distinction between the "target" and "host" architectures is no longer necessary. Therefore, these have been merged into a single architecture property. The effects of this merge are as follows:

John Reppy

Fix for bug #230 (New literals-lifting code does not handle pair of reals).

John Reppy

Simplified the runtime-system build rules for Cygwin.

John Reppy

Created the script config/prepare-win-install.sh, which implements the fetching and unbundling of source and bin files in preparation for a Windows installation.

John Reppy

Fix for bug #229 (Real.fromString errors). This bug was actually two unrelated issues. The problem that Real.toString returns Real.posInf for 0.0e123213213123213123123 has been fixed in the RealScan module (system/basis/Implementation/real-scan.sml). The second bug was a regression introduced in 110.93, where the SIGFPE signal was specified as the result of the into instruction, whereas Linux actually signals SIGSEGV for into. Note that there may be a related issue of BSD systems, where SIGBUS might be the signal, but we need access to a test machine to verify.

John Reppy

Fix for bug #230 (segmentation fault when compiling MLton sources with SML/NJ 64-bit). The problem was that when a large vector was being created, the assembly code did not correctly restore the stack state before trying to call the runtime system to do the allocation.

John Reppy

The runtime now uses MAP_ANON for allocating memory on 64-bit Linux. This change fixes a problem with versions of Linux that do not allow access to /dev/zero (such as on ChromeBooks).

John Reppy

Modified the generic installed (base/system/smlnj/installer/generic-install.sml) to support conditional targets. You can now write tests like

The symbols that can currently be tested for are SIZE_32, SIZE_64, UNIX, and WINDOWS. See the config/targets file for more details.

John Reppy

Fixed bug #227 (CPS contraction is taking an excessive amount of time on word8 basis test).

John Reppy

Modified the CPS contraction phase to optimize the case where a numeric conversion is applied to a constant value.

John Reppy

Modified the Unix installer script (base/system/smlnj/installer/nix-install.sml) to pass a size argument to the configuration script. This argument is used by the ASDL configuration.

John Reppy

Overhauled the installation script (config/install.sh) and various script templates (e.g., config/_run-sml) to allow setting the default size. The config/install.sh script now supports the following arguments:

It is possible to install both versions in the same location by running the install.sh script twice. For example, the commands

will install both versions with the 64-bit version as default. One would then use the command sml -32 to run the 32-bit version of the system. Note that the default version must be installed second.

John Reppy

Added support for the -64 flag to the fixpt script in base/system.

John Reppy

Added support for the -64 flag to the cmb-make script in base/system.

John Reppy

Renamed the REAL representation constructor to Raw64, which matches what is going on in the runtime system. Also renamed the toReal function to toReal64.

John Reppy

Updated the SMLofNJ.SysInfo structure by removing constructors from the oskind datatype that correspond to obsolete systems. Also added a getHostSize function that returns the host architecture’s native word size in bits (e.g., 32 or 64).

John Reppy

Added the -64 flag to the testml script in base/system and to the .run-sml script. Thus, one will be able to specify the 32-bit version of SML/NJ using the command sml -32 and the 64-bit version using the command sml -64. Currently, 32-bits is the default, since the 64-bit system is unstable.

John Reppy

Removed obsolete operating systems from the SMLofNJ.SysInfo.os_kind datatype. This change reduces the type to two constructors: UNIX and WIN32. Also added a function getHostSize to the SysInfo structure, which returns the host word size in bits (i.e., either 32 or 64). The word size is now reported in the compiler’s banner message at startup.

John Reppy

Fixed bug #130 (failure to raise Bind exception). Added a function refutable to ElabData/types/typesutil.{sig,sml} and used it to limit type generalization of val bindings in Elaborator/types/typecheck.sml. The fix does not deal properly with refutability of OR patterns, but OR patterns in val bindings is a dubious feature.

This change also fixes bug #188 (Missing warning for nonexhaustive valbind patterns), bug #190 (Unexpected exception in SML/NJ with invalid list pattern match), and #199 (Compiler bug in pretty printing of result).

David MacQueen

Modified the cmb-make script to support passing compiler control flags to the build command. The flags should be specified after the path to sml command (if it is given).

John Reppy

Finished the implementation of the new literal bytecode engine. There is a control flag (Control.CG.newLiterals that allows switching between the old and new bytecodes).

John Reppy

Fix for bug #225 (Math.ln giving erroneous answers on Windows). The problem was an inconsistency in the way the Unix and Microsoft assemblers interpreted the addressing mode for the FLD instruction.

John Reppy

Clean up in the Basis Posix library code (both SML and runtime) to be consistent about when the SysWord.word type is being used to communicate information between SML code and the runtime system.

John Reppy

Add support for specifying a 32 or 64-bit target as command-line option to the .arch-n-opsys and .link-sml scripts. The default size is currently 32-bits, but that will change once 64-bit support is solid.

John Reppy

Generalize code generation for conversions involving tagged integers/words, where the size is not the default integer size. This situation only occurred for Word8.word values on 32-bit targets, but also occurs for 32-bit values on 64-bit targets.

John Reppy

Rewrote the expansion of the INLLSHIFT, INLRSHIFTL, and INLRSHIFT primops (compiler/FLINT/trans/transprim.sml). The expansion process now correctly handles shift operations on types that are smaller than the default tagged-integer size. This change also allows the Word8 shift operations to be inlined.

John Reppy

Fixed a bug in the constant folding of arithmetic-right-shift operations. The sign was not getting extended for words when the most-significant-bit was set.

John Reppy

Fixed a bug in the Real.toManExp function (the exponent was off by one, which meant that the mantissa was two times its expected value). This fix also fixes a problem in Real.toLargeInt, where the function would go into an infinite loop in some cases.

John Reppy

Fixed bug #173 (OS.Process.sleep only works with whole numbers). For systems that have finer-grain sleep function, such as the nanosleep(2) system call, the OS.Process.sleep and Posix.Process.sleep functions now support sub-second granularity.

John Reppy

Restructured the CPS contraction phase to make the fusion of integer/word conversions more uniform. Also fixed a bug where Int32.fromLarge(Word32.toLargeInt 0wxffffffff) would return ~1 instead of raising Overflow. The problem was that TEST(m,n) o COPY(n,p) was getting fused to COPY(m,p) when m = p, instead of TRUNC(m,p).

John Reppy

Int64 comparisons were not always correct, which lead to some positive values being printed as negative numbers (basically when the sign bit of the lower word was set).

John Reppy

Added Unsafe.IntInf structure, which provides access to the internal representation of the IntInf.int type. Note that this representation may change in the future.

John Reppy

Fixed bug #223 (Incremental Build fails on Windows). There was a missing CloseHandle() when getting a file’s timestamp.

John Reppy

Changed base/system/allcross script to use cmb-cross script. Also modified the cmb-cross script to build compressed tar files, when given the -z option, and to clean up intermediate files.

John Reppy

Restructured the amd64 machine-code generation implementation and filled in many of the missing encodings. It should be complete for SML/NJ code generation, but needs more work to support the full set of operations described in the amd64.mdl file.

John Reppy

Some cleanup in the x86 MLRISC backend. Removed the MULB, MULW, and MULL unsigned-multiplication instructions, since they are not binary operations. The MULL instruction is covered by the MULL1 constructor in the multDivOp datatype. The same change was applied to the amd64 backend.

John Reppy

Many changes to the amd64 machine description:

John Reppy

New script for cross compiling to other architectures; the script is still called cmb-make, but now supports target-specific dependencies in the front-end (i.e., representation of numeric types and endianess). The cross compilation scheme was developed by Matthias Blume and then encoded in a script.

John Reppy

The runtime system now builds for the amd64 architecture. Most of the changes relate to the difference between the flat BIBOP on 32-bit platforms and the two-level BIBOP on 64-bit platforms.

John Reppy

Fix bug #224 (Word64.fromLargeInt fails). The problem was an incorrect record kind in CPS/opt/infcnv.sml (it was RK_RECORD instead of RK_RAWBLOCK).

John Reppy

Changed the rep datatype constructor Word32 to Raw (which covers both 32 and 64-bit numbers on 32-bit platforms). We now check the length of the raw object when converting to an concrete numeric type.

John Reppy

Removed the use of runtime-type passing for polymorphic arrays. The effect of this change is that code that uses the Array.array type will be faster when the element type is not real (e.g., sorting an Int32.int array was 1.2 times faster), but slower when the type is real. Use the monomorphic type RealArray.array for best performance on arrays of reals.

Dave MacQueen and John Reppy

We added a new primop, REAL_TO_BITS that casts a floating-point value to the same-size word value. This primop allows the Assembly.logb function to be implemented in SML.

We have also refactored the implementation of the Math structure to share common code across the versions that are specialized for different levels of hardware support.

John Reppy

Rewrote the assembly code for the x86 and AMD64 targets. Previously, there were separate source files for Unix and Windows; these have been replaced by a single common file (one for each architecture). The assyntax.h file has also been replaced by x86-syntax.h, which covers both the x86 and AMD64 on both UNIX and Windows.

The AMD64.prim.asm file now compiles, although there are a few minor issues that will have to be fixed once we have a working code generator. We have also fixed a number of issues in the garbage collector related to the use of the 2-level BIBOP on 64-bit targets.

John Reppy

Some cleanup in the interval-timer code. In keeping with the other time-specific functions, I have switched the runtime-system API to use unsigned 64-bit nanoseconds to specify time values. I have also added an implementation for c-libs/smlnj-runtime/itick.c, which was missing. Lastly, moved the Windows-specific file win32-timers.c from runtime/kernel to runtime/mach-dep.

John Reppy

Added 64-bit implementations of the target-specific Basis Library modules in directory Basis/Implementation/Target64Bit.

John Reppy

Added PackWord64Big and PackWord64Little structures to Basis Library. Note that the implementation of these is target-specific.

John Reppy

Added bigEndian flag to the TARGET signature.

John Reppy

Fixed the Concurrent ML library to use 64-bit positions (both Unix and Windows) versions.

John Reppy

Moved the year offset from SML to the runtime system. This change is necessary because Windows uses 1601 as year 0, whereas UNIX uses 1900. We have also switched to using unsigned 64-bit times in nanoseconds as the interface between the Basis code and runtime system. This change is consistent with the other places where time values are communicated between the runtime and SML code.

John Reppy

Fixed a problem with CM’s symbol filtering (see bug #222).

The problem could manifest itself when a library l2.cm imported two symbols A and B from l1.cm and then exported the same A but a different B (which could have been defined in terms of the imported B). Moreover, for the problem to occur both A and B within l1.cm must have come from the same SML source file.

With the above setup, when running

Various 64-bit porting changes to the Windows implementation of the Basis Library and runtime system:

Implemented Basis Library proposal 2019-001 (Correction to the PRIM_IO signature). This proposal changes the return type of the avail function in a reader to be Position.int option, which is necessary to support large files.

John Reppy

Added primop support (PTR_TO_WORD and WORD_TO_PTR) for the c_pointer type that was added in 110.89. These primops are exposed in the new InlineT.Pointer structure. We define a PointerImp structure that is used inside the Basis implementation and a Unsafe.Pointer structure that is visible to users.

John Reppy

Switched the Position structure to be bound to Int64 and updated the runtime system to use 64-bit integers for file offsets and time values (in nanoseconds). This change fixes bugs #33 (Overflow exception with inputLine function) and #36 (Can’t open very large file).

John Reppy

Added abstract c_pointer type to the primitive types. This type will be used to represent runtime-system pointers (e.g., the HANDLE values in the Windows implementation).

John Reppy

Removed makefiles and code for architectures and operating systems that are no longer supported (e.g., the DEC Alpha and HPPA architectures).

John Reppy

Switched the FixedInt and LargeWord structure aliases to be 64-bits (i.e., FixedInt is now bound to Int64 and LargeWord is bound to Word64).

John Reppy

We are now assuming that we have at least C99 support (for practical purposes, this assumption is even true on Windows). With this assumption, the allocation of small objects in the runtime has been switched from macros to inline functions (see runtime/include/ml-objects.h). This change allows a graceful handling of 32-bit integers, which are heap allocated on 32-bit machines, but tagged on 64-bit machines.

John Reppy

Fixed various bugs in the implementation of the Word64 operations. The addition and subtraction operators were using arithmetic right shifts, instead of logical right shifts. Also, the translation of 64-bit shift operations was incorrect because of a typo in the variable names.

John Reppy

Created a simplified version of the MLRiscGen functor. This version of the functor, which is in the file CodeGen/main/mlrisc-gen-fn.sml does not include the memory disambiguation and GC types code. Since the old version (CodeGen/main/mlriscGen.sml) did not use these features by default, there should be no difference in the quality of the generated code.

The purpose of this change is to remove unused code that has 32-bit dependencies.

John Reppy

Added contraction for unsigned REM and NEG operations in CPS/opt/contract-prim.sml.

John Reppy

Moved the compiler/DEVNOTES directory to the dev-notes tree and renamed it old-compiler-notes.

John Reppy

Added 64-bit versions of NumFormat and NumScan. We use the 32-bit version for numbers of 32-bits or less and the 64-bit versions for numbers with up to 64 bits. Thus, on 32-bit machines, the default int and word types use NumFormat32 and NumScan32, while on 64-bit machines they use NumFormat64 and NumScan64. This change also required splitting out some common code into a ScanUtil structure and also splitting out the scanning of real numbers into the ScanReal structure (formatting of reals was already in its own structure).

John Reppy

Reimplemented the 64-bit int and word types to put them on a (mostly) equal footing with the other precisions. In this new implementation, the basic types int64 and word64 are now PRIMITIVE (instead of being ABSTRACT type represented by pairs of boxed 32-bit words). Arithmetic and comparison operations on these types are represented as primops and are preserved as such up to just before closure conversion. At that point, the new Num64Cnv structure (compiler/CPS/opt/numcnv.sml) is used to expand 64-bit operations and constants into 32-bit operations. Most of the 64-bit primops are inline expanded, but multiplication and division operations are converted to calls to library code from the CoreInt64 and CoreWord64 modules (system/smlnj/init).

Because the type are primitive, we were able to change the runtime representation to use packed records (RK_RAWBLOCK) to represent them, which saves space and should also help with performance.

See the dev-notes/num64.md file for more details about the implementation.

John Reppy

Reorganized the Basis Library source files (system/Basis) to isolate dependences on target word size.

In the Basis/Implementation directory, I created subdirectories (e.g., Target32Bit) to hold implementations that are specific to the target. These directories include a bind-structs.sml file that replaces the many bind-*.sml files in Basis/Implementation.

In the Basis/Exports directory, I replaced the many individual files (each with a single module renaming) with bind-common.sml (for target-independent bindings) and a target-specific file (either bind-target-32-bit.sml or bind-target-64-bit.sml).

John Reppy

Some of the CPS optimization modules (Expand and EtaSplit were written as functors over the machine spec, when, in fact, they never reference their functor argument. Therefore, they have been converted to structures.

John Reppy

We now use the InlineT.identity primop for Fn.id, so the compiler can optimize it.

John Reppy

Fixed pretty-printing regression in 110.87; value of char type were missing their enclosing quotes.

John Reppy

Made the Char.chr operator inline (a primop was added to support this change in 110.86).

John Reppy

Major renaming of the primitive operators in the Inline structure (as described in dev-notes/primop-list.md). Also cleaned up the Basis Library implementation to remove most (but not all) 32-bit dependencies.

John Reppy

Added cases to the top-level pretty printer to handle the new basic types that were added in 110.86 (e.g., word8vector and chararray). Also changed the way that primitive types are handled to use a table keyed by tycons, instead of a sequence of nested conditionals.

John Reppy

Added word8vector and chararray to the primitive types that the compiler knows about. These will be used in the rewriting of the InlineT structure.

John Reppy

Replaced the Primop.primop constructors NUMSUBSCRIPT and NUMUPDATE with

`sml | NUMSUBSCRIPT of numkind | NUMSUBSCRIPTV of numkind | NUMUPDATE of numkind | INLNUMSUBSCRIPT of numkind | INLNUMSUBSCRIPTV of numkind | INLNUMUPDATE of numkind `

This design matches the naming conventions for polymorphic subscripting and updating.

John Reppy

Added Primop.INLCHR to implement Char.chr as an inline function. This change also required moving the definition of the Chr exception to the Core module so that it is accessible to the translate phase. The inline version of Char.chr will be enabled in the 110.87 release (we need the internal primop before we can use it).

John Reppy

Major overhaul of the representation of primitive operators (both in the Primop and CPS.P structures). The primitive arithmetic and comparison operations are now defined in the ArithOps structure (ElabData/prim/arithops.sml). There are three datatypes defined in this module

These types are used in both the Primop and CPS.P modules, which makes the translation between representations more direct.

Some details:

John Reppy

Removed unused record kind constructors (RK_SPILL, RK_EXN, and RK_BLOCK) from CPS.record_kind datatype. Also renamed RK_I32BLOCK to RK_RAWBLOCK and RK_FBLOCK to RK_RAW64BLOCK. Various other renamings to remove 32-bit assumptions.

John Reppy

Renamed DTAG_raw32 to DTAG_raw, since the semantics on 64-bit systems will be to require word-size aligned raw data. Also renamed ML_AllocRaw32 to ML_AllocRaw and ML_ShrinkRaw32 to ML_ShrinkRaw for similar reasons.

John Reppy

Removed unused flags from the Control structure; most of these came from Control.CG, where roughly 20 out of 60 flags were no longer used.

John Reppy

Split the contraction of primitive operators out of the Contract structure into its own ContractPrim structure.

John Reppy

Split the translation of primops to PLambda out into its own file (compiler/FLINT/trans/transprim.sml).

John Reppy

Fixed regression: Word32.toInt 0wx8002DE32; would return 187954 instead of raising Overflow. The problem was a mistake in the way that the overflow trap was being generated in MLRiscGen.

John Reppy

Some minor primop cleanup.

John Reppy

Improvements to the core 64-bit int and word modules in system/smlnj/init. Replaced Int64.+, Int64.-, Word64.+, and Word64. with versions from *Hacker’s Delight that use fewer conditional branches. Also replaced the relational operators (<, ⇐, etc) with more direct implementations.

John Reppy

Fix for bug #213 (Int32.div raises Div instead of Overflow when dividing minInt by ~1). Since the compiler generates an explicit test for division by zero, we know that the only arithmetic traps must be caused by other operations. Therefore, we can just map any arithmetic trap to Overflow.

Also removed the old SPARC assembly code for multiplication and division. The code generator always uses the native hardware instructions, so the assembly code is not needed.

John Reppy

Yet another attempt to get the implementation of use in the REPL working in a sensible way.

With these changes, use should behave as follows. If an invocation of use encounters a compilation error (either in the initial file or in a nested invocation of use), then the compiler error message will be printed and the call to use will immediately return (). If an invocation of use raises an exception during execution of the compiled code (either in the initial file or in a nested invocation of use), then the exception will be reported at the top-level. Any change to the global state or environment that occurs before an error is encountered, will not be rolled back.

Files specified as command-line arguments to the sml command will be treated as if use was invoked on them. If there is an error, then the error will be reported and the sml command will terminate with a non-zero exit status (at least on Unix).

This change fixes bugs #193, #217, and #219. There is a connection between this change and #183, which was fixed in Version 110.82.

John Reppy

Change to the CPS primops: moved the F_SGN operator (which is unary) from the fcmp datatype to the branch datatype (and renamed it FSGN).

John Reppy

Finished conversion of the CPS IR to a form that is compatible with ASDL. Basically, this involved converting the datatype constructor names to upper-case identifiers.

These changes are a step in the plan to eventually switch to an LLVM-based code generator that will be given pickled CPS code as its input.

John Reppy

Starting to migrate the CPS IR toward the ASDL version. Changed the names of the CPS.P.arith and CPS.P.cmpop constructors to be upper-case alpha IDs (many of them were symbolic identifiers). Also split out the various utility functions into the new CPSUtil module (CPS/cps/cps-util.sml). Lastly, moved the literals.sml file from FLINT/main to CPS/main (where it belongs).

Note that the CPS.P.arithop datatype is now identical to the Primop.arithop datatype

John Reppy

Reorganized the backend of the compiler by moving the CPS-related code into its own directory tree (Compiler/CPS) and replacing the FLINTComp functor with the FLINTOpt structure and the CPSCompFn functor. The conversion from FLINT to CPS is part of the CPSCompFn functor, which takes the program representation all the way from FLINT to machine code segments.

John Reppy

Modified config/install.sh to look for a pre-Mojave SDK when trying to install on macOS 10.14 Mojave.

John Reppy

Updated runtime/objs/cygwin.def so that the runtime system will build on 32-bit Cygwin. Also updated installation script to suggest using the 32-bit version of Cygwin when a user tries to install it on Cygwin64.

John Reppy

Xcode 10.1, which is Apple’s development environment for macOS 10.14 Mojave, does not include the libraries needed to build 32-bit executables, such as the SML/NJ runtime, although 32-bit programs will still run.

To support building on Mojave, I added a new Makefile (mk.x86-darwin18) for the runtime system and modified the config/install.sh script to use this makefile when necessary. This new makefile expects that the MacOSX10.13.sdk directory from Xcode 9 has been copied into the Xcode 10 SDKs directory. Note that updating Xcode from the AppStore will likely remove the 10.13 SDK, so you should keep a copy in a safe place.

The Xcode SDKs live in Platforms/MacOSX.platform/Developer/SDKs under the Developer directory. One can determine the path to the current developer directory using the command

John Reppy

Removed several unsupported primitive operators from the compiler. In the CPS IR, these were free, acclink, setpseudo, setmark, and getpseudo. The pseudo-register operations were not supported in the code generator, while the others were no-ops. The corresponding operators GETPSEUDO, SETPSEUDO, SETMARK, and DISPOSE were removed from ElabData/prim/primop.sml and their bindings were removed from Semant/prim/primop-bindings.sml and the InlineT and Unsafe structures.

The AllocProf module in the compiler was also disabled, since it relied on the pseudo registers for recording profile information at runtime. Furthermore, uses of the acclink primitive operation in FLINT/cps/closure.sml when static profiling is enabled were removed.

These changes were committed as revision 4886.

John Reppy

Fix for bug #216 (run-time system fatal error with large top-level value). The problem was in the code for building literals.

John Reppy

Change CPS operators for wrapping/unwrapping integer and float values to be word-size flexible. We now use a single wrap (and unwrap) operator that is parameterized by a numkind value. We also changed the wrap/unwrap operators to box/unbox. The mapping from old operators to new ones is as follows:

John Reppy

Further cleanup for 64BIT in function atomeq in PEqual. (base/compiler/FLINT/trans/pequal.sml). Added numKind, intEqTy, and uintEqTy functions. The numKind function should be extended once int64 and word64 are treated as primitive types in the compiler.

Dave MacQueen

Fixed 64BIT issue in module MatchComp (base/compiler/FLINT/trans/matchcomp.sml). Added int64Ty and word64Ty cases to function numCon.

Dave MacQueen

Fixed 64BIT issue in module Equal (base/compiler/FLINT/reps/equal.sml). Exports just one function: equal_branch, which is called once in reps/wrapping.sml to type-specialize branches on calls to POLYEQUAL.

Dave MacQueen

The CPS optimizer had a mechanism for checking the CPS against the FLINT types, which required maintaining a mapping from lvars to their FLINT types. This code has long since bit-rotted and cannot even handle a simple expression like 1+2. Therefore, I’ve removed the mapping (a hash table) from the CPS optimizer and the vestigial code that modified it in the various CPS optimization passes.

John Reppy

Modified the InfCnv (now named IntInfCnv) structure to remove 32-bit dependencies.

John Reppy

Modified Pequal (in base/compiler/FLINT/trans/pequal.sml) and Translate (in base/compiler/FLINT/trans/translate.sml) to remove 32-bit dependencies. though further changes will be required to properly handle int64 and word64 types when defaultIntSz = 64.

Dave MacQueen

Reimplemented the Switch module (int base/compiler/FLINT/cps). The new implementation follows the same basic design as before, but the code is better organized and documented, and it now uses the concrete CPS representations, instead of being parameterized over an abstraction of them. It also now uses binary search for boxed (e.g., Int32.int) switches.

John Reppy

Reimplemented the array/vector-slice modules to use a (base, start, length) representation (as does Substring in system/smlnj/init/substring.sml). Also fixed a bug in the slice findi functions, where the index being passed to the predicate function was not adjusted to be slice-relative.

John Reppy

Implemented Basis Library proposal 2018-002 (Additional slice operations).

John Reppy

Improved implementation of CharVectorSlice.map and CharVectorSlice.mapi to not build intermediate list of results.

John Reppy

A beta-release of ASDL library and asdlgen tool have been added to the system. This version of the tool implements SML support, but the C++ support is not complete. There is a CM tool for ASDL, which recognizes the .asdl file suffix.

John Reppy

Two changes to the installer (base/base/system/smlnj/installer):

John Reppy

Added RENAME extension style to CM tool support. This extension style allows arbitrary file names to be generated from the base name.

John Reppy

Fixed a bug in the implementation of monomorphic buffers: the functions CharBuffer.add1 and Word8Buffer.add1 had an incorrect length test.

John Reppy

Fixed a compiler bug (arg ty lists wrong length) in unifyTy that could occur when one of the type constructors is the ERRORtyc. This bug occurs because the ERRORtyc is equal to any other type constructor, which (incorrectly) implies that the number of type arguments should be equal.

John Reppy

Fixed #206 (Parsing of explicit type variables and val rec is broken). This bug was also bug number 1261 in the old bugs list.

John Reppy

Fixed minor bug in Date.toString (missing leading "0" for day of month). This issue was bug number 1444 in the old bugs list.

John Reppy

Cleaned up match compiler code (FLINT/trans/matchcomp.sml) and added typing and function comments. Added debugging and printing infrastructure, including new FLINT/trans/mcprint.sml file, and new Control.MC.debugging flag.

Dave MacQueen

Fixed parser to allow parentheses around val rec patterns.

John Reppy

Fixed the scanner to produce the correct error message for bad escape sequences in string literals.

John Reppy

Fixed old bug number 1383: Char.toCString #"\000" returned "\\0", instead of "\\000", which caused String.toCString to produce invalid results.

John Reppy

Fix for bug #201 (The AMD64.cm library is missing).

John Reppy

Added MONO_BUFFER signature, with instances CharBuffer and Word8Buffer, to Basis implementation ( Basis Library Proposal 2018-001).

John Reppy

Fix a bug where “0w” was being accepted as a prefix for a hexidecimal word value in Word.fromString/scan (ignoring case, only “0x” and “0wx” are valid prefixes). This change fixes bug number 1375 from the old bugs list.

John Reppy

Fixed a bug in the parsing of bindings involving the op keyword. The parser was more restrictive than the definition. This change fixes bug number 1370 from the old bugs list.

John Reppy

The lexer gave an unmatched close comment error on "*)", when it should have scanned it as the tokens "*" ")". This change fixes bug number 330 in the old bugs list.

Note: there is some ambiguity as to what the correct behavior should be here. The Definition of Standard ML (1997) only says that unmatched open comments should be signalled as errors, but the Commentary on the Definition of Standard ML (1991) says otherwise in Appendix D. SML/NJ started signalling an error in version 0.71, but we choose to revert to accepting this sequence, to match the 1997 Definition (and the behavior of other systems).

John Reppy

The sameSign function returned incorrect results in the Int31 and Int32 modules.

John Reppy

Fixed various minor parsing and scanning issues:

John Reppy

Completed overhaul of the way that int/word literals are handled in the compiler. We now use IntInf.int to represent the values in all IRs. This change also results in better CPS contraction, since we now perform constant folding for both signed and unsigned values at all sizes. We were also able to get rid of the tricky code that worries about large tagged integer values that might cause overflow during code generation.

John Reppy

Improved the reporting of errors involving literal values. We now use the original source text when describing the value in the error message.

John Reppy

Fix for bug #191 (Compiler crash when handling large reals). We now issue a warning for real literals that will round to zero and an error for real literals that are too large to represent. There still needs to be done some work to support sub-normal literal values (these are currently rounded to zero).

John Reppy

Changed the representation of real literals from strings to RealLit.t.

John Reppy

Removed real patterns from Absyn and FLINT, since they are not allowed by SML'93 and were not present in the AST representation.

John Reppy

Fix for bug #194 (Real.fromString overflows or hangs). There were two issues here. First, the Overflow exception was being raised when scanning large exponents, but it was not being handled by the scanning code. The second issue was that the scaling loop for large exponents did not immediately terminate once infinity (or zero) was reached, so it could take a long time.

John Reppy

Moved the Version-1 literal building code into gc/old-literals.c. This file can be removed once the compiler generates the Version-2 literal bytecode.

John Reppy

Moved the check for whether a int or word literal is in range for its type from the absyn→plambda translation to the overload resolver (compiler/Elaborator/types/overload.sml).

John Reppy

Part 1 of an overhaul of the way that the compiler treats int/word literals. The end goal is to use IntInf.int to represent literals throughout all phases of the compiler. In this step, we changed the representation of literals in the Absyn representation (earlier representations already used IntInf.int).

John Reppy

Fixed unnumbered bug in IntInf.mod and IntInf.rem functions, where the Div exception was not getting raised when both arguments are 0.

John Reppy

Various bits of cleanup in the handling of primitive operations, such as removing the ptnum mechanism for translating from Absyn to FLINT.

David MacQueen

Added Target module, which specifies the properties of the target (e.g., the size in bits of the default int type). Reworked the generation of the InlineT structure to be target specific.

John Reppy

Removed FLINT primops (and their CPS counterparts) that are not in the InlineT structure and, thus, are never used by the compiler.

John Reppy

Fixed bug #123 (missing nonexhaustive bind warning). The mkVBs function in FLINT/trans/translate.sml was adding a redundant default rule by calling ElabUtil.completeMatch after a default rule had already been explicitly added to the match for let bindings.

David MacQueen

Fixed bug #183 (status code returned by sml REPL). This fix restores the version 110.79 behavior of having sml foo.sml exit with a non-zero status when there is a type-checking error in foo.sml. It also cleans up the error messages associated with use when there is a syntax error.

John Reppy

Fixed bug #185 (Bring command line help text into parity with man page). Added missing options (@SMLversion and @SMLsuffix) to the help message that is printed for the command “sml -h”. Also adjusted the order of options in the help message, and in the man page, so that the orders match.

John Reppy

Changed the way that we test for allocation-space addresses in minor GCs. Instead of using the BIBOP, we now do a pointer range test. On 32-bit systems, this change results in a small (~0.13%) performance boost, but we expect a bigger impact on 64-bit hardware, where the cost of BIBOP probes will be higher and there are more registers available to hold the nursery bounds.

John Reppy

Fixed some issues in build-literals.c. These were mostly false positives in the assertions, but there was also a bug in the way that the available space was tracked that could conceivably result in a crash (but was very unlikely).

John Reppy

Updated _arch-n-os script to recognize macOS 10.13 (High Sierra) as a valid target.

John Reppy

Fixed a bug in the way that JSON string values were being printed. The code previously assumed that C-style escaping will work, but that is not true for "\'" (as well as for control and non-ASCII characters). The new implementation assumes that the string value is UTF-8 and uses the "\\u" escape sequences for characters outside the JSON escapes and printable ASCII characters.

John Reppy

Fixed bug #129 (Symbolic identifiers are allowed as strids).

Dave MacQueen

Fixed bug #179 (ml-ulex writing debug messages to stdOut). Both ml-ulex and ml-antlr now direct their debug and status messages to stdErr (instead of stdOut).

John Reppy

Linux distributions are starting to require that the stack be marked as non-executable in applications. Because the runtime system includes assembly code, this marking was not happening. We’ve added .section directives to the PPC.prim.asm and X86.prim.asm files as per https://wiki.gentoo.org/wiki/Hardened/GNU_stack_quickstart#Patching. Thanks to Daniel Moerner for reporting this issue and for providing a pointer to the fix.

John Reppy

Added --debug command-line option to ml-antlr. This flag causes <b>ml-antlr</b> to generate debug actions that print the left-hand-side non-terminal of the production.

John Reppy

Working on 64-bit support. Changes include making code generation dependent on the target word size and abstracting over the BIBOP representation in the runtime system.

John Reppy

Further cleanup of the separation of FLINT from the front-end. Eliminated all references to ModulePropLists (module-plists.sml) in the front end and in pickling, and moved module-plists from Semant/modules to FLINT/trans. ModulePropLists is now only used in FLINT/trans/translate.sml.

Revision: 4314

Files changed:

Dave MacQueen

Eliminated dependency of PlambdaType from the front end by adding a type TKind.tkind which is a simplified standin for PlambdaType.tkind for use during elaboration. TKind.tkind values are translated on demand to PlambdaType.tkind in trans/transtypes.sml. Types still has a tycpath type but it is defined using TKind.tkind now. The new structure SigPropList replaces ModulePropLists (Semant/modules/module-plists.sml) for use in instantiate.sml. Instantiate is now defined directly as a structure so the functor application in Semant/modules/instantiate.sml no longer exists.

Files changed:

Dave MacQueen

Added support for Successor ML record-expression-punning syntax. For example, one can now define a function f as

which is equivalent to the definition

John Reppy

Fixed a bug in the parser. Asterix (*) was not allowed as a record label when using the record-pattern-punning syntax.

John Reppy

Added support for do exp Successor ML syntax.

John Reppy

Fixed bug #153 (Enabling Successor ML features is delayed). We now use a function Control.setSuccML to switch to/from Successor ML mode in the REPL. The function resets the parser, so the next input will be correctly parsed. The Control.succML flag is no longer visibile in the REPL.

John Reppy

Fixed bug #149 (Datatype replication exposes hidden constructors). Added boolean field stripped to DATATYPE variant of tyckind in compiler/ElabData/types/types.sml with default value false. stripped is set to true when a datatype is matched with a simple type spec in signature matching, and datatypes with stripped set to true are disallowed in datatype replications.

Files changed:

Dave MacQueen

Added %tokentype directive to ml-antlr; this directive allows users to specify the token datatype externally, which is necessary in order to share a lexer with two different ml-antlr parsers.

John Reppy

Change the interface to AMD64Gen in MLRISC; the signBit and negateSignBit callbacks now return an MLTree.rexp (instead of a label).

John Reppy

Fixed #151 (Error installing from source on Mac OS X). The fix involves both changes to the config/install.sh script and the mk.x86-darwin makefile. With this fix, we include the SDK argument to the /usr/bin/as only when the OS version is 10.10 (Yosemite) or later.

John Reppy

Added the proposed unzipMap, unzipMapi, find, and findi functions to the ListPair module.

John Reppy

Added the proposed mapLeft, mapRight, appLeft, and appRight functions to the Either module.

John Reppy

Fixed bug #145 (Internal exception occurs on bogus annotation instead of typechecking diagnostic). Added missing OVLD_UB case in function failMessage in compiler/Elaborator/types/unify.sml.

Dave MacQueen

Fixed bug #166 (Can’t install SML/NJ in directories containing spaces). Thanks to Eugene Sharygin for the patch.

John Reppy

Fixed incorrect dividend sign extension before 32-bit divide in amd64 code generator in MLRISC

Mike Rainey

Fixed bug #150 (Add title to batch script).

John Reppy

Implemented the changes for Basis Library Proposal 2016-001. This proposal added the popCount function to the WORD signature.

John Reppy

Fixed bug #156 (sml resumes after SIGSTOP with bogus exception report). The fix is a bit of a hack: I modified the non_bt_hdl function in evalloop.sml to match an IO.Io exception with the appropriate shape for this situation.

John Reppy

Fixed bug #154 (Return code for ml-ulex when there is an error).

John Reppy

Fixed bug #155 (Misleading printing of word literals in error messages).

John Reppy

Fixed a bug in the implementation of the --ml-lex-mode flag for ml-ulex. The \h escape sequence is supposed to map to the character range [\128-\255], but did not.

John Reppy

Fixed bug #147 (Hexadecimal escapes in strings are not supported). We previously did not support Unicode escapes in string literals. We now do so, with non-ascii codepoints being mapped to the UTF-8 encoding with escape values in the range 0..255 being mapped to the corresponding 8-bit character. Values outside that range are flagged as an error.

Revised August 4, 2016

John Reppy

Partial fix for the noisy exception-stack traces on the Error exception. The cases that are handled by this change are applying use to a non-existent file and when there are compilation errors in a program being built by CM.make. What remains to be handled is the situation where CM.make is applied to a non-existent file.

John Reppy

Patched base/compiler/FLINT/clos/closure.sml so that Twelf will build again. Fixes bug #140 (Lookup failure in closure.sml when compiling Twelf).

John Reppy

Added support for a Successor ML tool to CM. This tool allows one to specify that a source file fool.sml is Successor ML source code in the following ways:

John Reppy

Added the directory base/old-basis to support backward-compatible views of the Basis Library. You can use these by replacing the line

with

in your CM files.

New implementation of Date structure in the Basis, which fixes bugs #138 (Incorrect behavior for Date.fromTimeLocal) and #139 (Date.date is broken). Note that some more thought should be given to the correct semantics of Date.date when dealing with offsets. For example, should an offset of +23 hours produce the same date as an offset of -1 hours? Currently our implementation produces different results (by a day) for these two situations.

John Reppy

Implemented the changes for Basis Library Proposal 2015-003. This proposal added operations to the following signatures:

and the following structures:

While it is very unlikely that these changes will break existing code, there are a a couple scenarios in which the code might break. Namely, when use of open introduces conflicts and when user code implements one of the affected Basis Library signatures. Both of these examples occurred in the SML/NJ source code; the former in the ml-yacc sources and the latter in the MLRISC sources.

John Reppy

Added the optional implementations of PackReal64Big and PackReal64Little. This addition addresses feature request #82 (Implementations of PACK_REAL missing). The implementation uses the approach suggested by Michael Sullivan.

John Reppy

Fixed bug #45 (Compiler bug in specialize phase). This bug was in compiler/FLINT/opt/fcontract.sml and was the result of a bad interaction between eta contraction and inlining. As part of the fix, I cleaned up the code in this part of FLINT a bit.

John Reppy

John Reppy

John Reppy

John Reppy

Fixed bug #136 (Incorrect raising of exceptions in Real.fmt and Time.fmt).

John Reppy

Added Either structure and EITHER signature to Basis implementation ( Basis Library Proposal 2015-002).

John Reppy

Fixed bug #135 (Fails to build on Linux PowerPC).

John Reppy

Added Linux 4.* kernels to the list of operating systems recognized by the .arch-n-opsys script (fixes bug #134).

John Reppy

Added Mac OS X 10.11 (El Capitan) to the list of operating systems recognized by the .arch-n-opsys script.

John Reppy

Added support for Successor ML lexical extensions. These can be enabled using the command-line option -Cparser.succ-ml=true or by the assignment

at the REPL. The extensions are as follows:

This change is the beginning of a program to add Successor ML feature to SML/NJ; See https://github.com/SMLFamily/Proposed-Definition-of-Successor-ML for more details.

John Reppy

Major revision of the machinery for overloading resolution for both operators (vars) and literals, now using a common mechanism. This fixes bug #52 by improving the error message when an overloaded operator is inconsistent with its context. Updated 23 files, including major changes in overload.sml, types.sml, unify.sml, elabcore.sml, typesutil.sml. The overload declaration is still used in pervasives.sml, where the order of the specified instances of an ordering determines the default interpretation (i.e., the first one).

The SCHEME and LITERAL forms of tyvars are replaced by a new OVLD form that tracks potential instantiations of the type of the overloaded vars or literals.

David MacQueen

Moved base/NOTES/HISTORY file to doc/src/changelog/HISTORY.txt and converted it to ASCIIDOC format. Have also moved the README files from base/READMES to doc/src/release-notes. These changes are part of a general effort to rationalize and improve the documentation of the SML/NJ system.

John Reppy

Preliminary cleanups before changes to overloading

Minor cleanup in Elaborator/elaborate/elabcore (function elabOVERLOADdec) and in ElabData/types/typesutil.sml (function matchScheme). Preparing for a new method of handling type checking of overloaded operators. [Note that there is no reason for the options field of OVLDvar to be a reference — it is never updated. Changing this requires corresponding change in pickling.]

Also added an etopdebugging flag (ElabControl) for debugging in elabtop.sml. Modified elabcontrol.{sml,sig} and elabtop.sml. Also rearranged ast and absyn printing in evalloop.sml.

David MacQueen

Improved error messages in ml-ulex for unclosed strings. Also made documentation improvements.

John Reppy

Added -D_FILE_OFFSET_BITS=64 flag to x86-linux makefile. This flag is necessary to avoid spurious EOVERFLOW errors on some versions of Linux. The problem appears to be limited to large file systems that have more than 2³² inodes.

John Reppy

Added %value directive to ml-antlr; this addition improves the error repair choices by allowing non-nullary tokens to be inserted when making repairs.

John Reppy

Created new doc tree in SML/NJ repository. Currently this tree just holds the sources for UNIX-style manual pages for the command-line tools (fixing bug #35). The documentation is written using the ASCIIDOC format. Use the following svn command to checkout a copy of the documentation tree:

John Reppy

Compiling the runtime system on cygwin was failing because the file exceptions.h was missing. It appears to have been part of previous versions, so a version has been incorporated verbatim in the file runtime/mach-base/cygwin-fault.c file (fixes bug #125).

John Reppy

Added the actionToString' and repairToString' functions to the AntlrRepair structure. These functions allow one to specialize the printing of tokens based on whether they are being added or deleted.

John Reppy

Added patches to support OpenBSD on PowerPC. The patches were contributed by Jasper Lievisse Adriaanse (fixes bug #124).

John Reppy

Use mkstemp to implement OS.FileSys.tmpName() on systems that support it (should be all modern versions of Unix). This change fixes bug #128. (Thanks to Johannes 5 Joemann).

John Reppy

Fixed a bug in IntInf.~>>, which did not handle negative arguments correctly (bug #110).

John Reppy

Fixed a problem in the CPS contraction phase. An optimization that eliminates construction of a record that already exists was not checking that the existing record was the same record kind (bug #119).

John Reppy

Switch to using MAP_ANONYMOUS to allocate memory on Linux systems. This change avoids problems when "/dev" does not support execute permission (as seems to be the case with some versions of Linux running on ChromeBooks; bug #120).

John Reppy

Fix for bug #127 (Crash on Windows with OS.Process.system).

John Reppy

Fixed a long-standing bug in Socket.recvVec, which prevented the result from being used in a string pattern match (thanks to Vesa Norrman for the fix).

John Reppy

Fixed minor issue in an error message; type variable name should be printed with leading '.

John Reppy

Added PackWord{16,32}{Big,Little} structures to the Unsafe module. This change makes the UNSAFE signature closer to the MLton version, although we still need to add the PackReal structures.

John Reppy

Fix bug #115 (BinPrimIO writer method getPos does not work under CML). Just needed to port the position update from mkReader code to the mkWriter code.

Lars Bergstrom

Fix bug #111 (Socket.acceptNB returns somewhat broken sockets). The problem was that under Win32, sockets returned from accept inherit their parents' non-blocking status, whereas on UNIX they are always blocking.

Lars Bergstrom

Fix bug #117 (BinIO.openAppend raises IO on non-existent file). We were opening the file for append if it existed but not creating it if it did not exist.

Lars Bergstrom

Fix bug #116 (Socket.sameDesc raises Match exception). The problem is that on Windows the iodesc datatype (defined in Basis/Implementation/Win32/pre-os.sml) has both an IODesc constructor and a SockDesc constructor. Updated the code in Win32/os-io.sml to handle the SockDesc constructor.

John Reppy

Fix bug #113 (Socket.select waits exactly twice the indicated timeout)

John Reppy

Fix AMD64 code generator to properly sign-extend arguments to IDIVQ.

Lars Bergstrom

Fix bug #108 (off-by-one error in Util/dynamic-array.sml; iterators crash)

John Reppy

Fix bug #107 (Bogus Int64 comparison operators)

John Reppy

Fixed bug #92. IntInf.scan now handles the “0x” prefix correctly. Also made minor improvements to the NumScan module.

John Reppy

Added Base64 module to SML/NJ Library to support encoding and decoding Word8 vectors as base64 strings.

John Reppy

Additions to the SML/NJ Library. Added exists, existsi, all, and alli functions to ORD_MAP signature and implementations, and added all function to ORD_SET signature and implementations.

John Reppy

Bug fix in ml-antlr to ensure that the generated toString function for tokens is strictly legal SML code (i.e., non-printing characters and UTF8 multibyte sequences are properly escaped).

John Reppy

Added getu function to ULexBuffer as a way to improve ml-ulex performance. This addition allows a fastpath for processing ASCII characters, which improved lexer performance by 3-4%.

John Reppy

Fixed bugs #89 and #96: Build Failure with Xcode 4.3 Also removed build support for MacOS X pre-10.5 (Leopard) on PPC and pre-10.6 (Snow Leopard) on Intel.

John Reppy

Fixed bug #88. The check for valid arcs on Unix systems now allows any character other than slash or nul.

John Reppy

Fixed implementation of Real.signBit on little-endian machines.

John Reppy

1) Fix for bug #60 recalculate strictness for DEFtyc`s in functor bodies when functor is applied (`Elaborator/modules/evalent.sml)

2) Fix for bug #77 separate ast representations for datatypes and datatype replications in decs and specs (multiple files)

3) set version to 110.74

Details in NOTES/changes/dbm_2012_1.

Dave MacQueen

Bug fixes for Unsafe.blastRead (bug #76): . proper error handling when reading from memory and there are not enough bytes. . pass correct data pointer and length to BlastIn (code was using old macros).

+ John Reppy

Added hash-table-based implementation of sets to SML/NJ Library.

John Reppy

Better error reporting under 32-bit linux for the missing dpkg support (bug #70). Enable 3.x kernels to build (bugs #80, #81, #83).

Lars Bergstrom

Added new S-expression library to SML/NJ Library (contributed by Damon Wang)

John Reppy

Fixed bug in JSON scanner (SML/NJ Library). It didn’t handle escaped backslash or double quote correctly.

John Reppy

Added boolean literals (true and false) to the conditional-expression syntax in CM. Thus, you can write

in a CM file. This change is meant to make it easier to use autoconf to configure the build process of an SML application.

John Reppy

Added missing String.scan function (bug #69). This also fixes the handling of certain corner cases by String.fromString.

John Reppy

Added the RTDSC and RTDSCP instructions to the amd64 code generator.

Mike Rainey

Added fix for comments in code bug (bug #63). Thanks to Michael Norrish.

John Reppy

Fixed bug in Socket.acceptNB (bug #59)

John Reppy

Fixed syntax error in ml-lex compatibility mode (bug #49)

John Reppy

Update _arch-n-opsys script for Mac OS X Lion (10.7).

John Reppy

Added Barriers module to CML.

John Reppy

Fixed ml-yacc examples to respect the changed signatures with respect to TextIO.inputLine.

Lars Bergstrom

Changed the Win32 implementation of validArc to support directories with extended characters (umlauts, etc.).

Lars Bergstrom

Fixed the Win32 socket and polling implementation to work correctly with CML. Signature of poll was wrong and didn’t handle sockets at all.

Lars Bergstrom

Fixed Real.toString and Real.fmt to include sign for negative zero.

John Reppy

Fixed the bug with Win32 calls to OS.Process.system not quoting the string.

Lars Bergstrom

Applied patch for building on more recent versions of NetBSD (bug #39).

Jon Riehl

Fixed performance bugs in List module by making @ and foldr be tail recursive (bug #51).

John Reppy

Fixed the Win32 unable to print long strings bug (bug #37).

Lars Bergstrom

Fixed an overrun during major GC. If the string arena was nearly full, it was possible for alignment padding added during copy to the to-space to overrun the allocated size.

Lars Bergstrom

The ml-antlr and ml-ulex programs have been ported to build under mlton.

John Reppy

Added %header directive to the ml-ulex scanner generator. Also updated the documentation.

John Reppy

Added @SMLsuffix flag to sml command. This flag can be used to get the suffix for heap files.

John Reppy

Added --strict-sml flag to ml-ulex for MLton compatibility.

John Reppy

Added %header directive to the ml-antlr parser generator. Also updated the documentation.

John Reppy

Changes to support compiling the runtime system on Mac OS X 10.6 (aka Snow Leopard).

John Reppy

Fixed a bug in the register-spill generator that is part of the MLRISC register allocator. The problem was that the code in RASpillWithRenaming functor assumed incorrectly that dedicated registers would appear in def/use information generated by ClusterRA. Thanks to Allen Leung for helping with this bug.

Mike Rainey

Removed redundant implementations of various top-level operations by consolidating them in base/system/smlnj/init/pervasive.sml. This change also fixes a bug in that the top-level version of round was incorrect.

John Reppy

Matthias Blume

Fixed bug in Int32.fmt when the argument was the minimum int and the radix was something other than DEC.

John Reppy

Fixed bugs in how ml-antlr parsed ML types in %tokens specifications.

Aaron Turon

Picking up some additional fixes for 110.69, including a fix for spaces in CM file paths.

Jon Riehl

Re-enabled some CPS optimizations (first_contract and eta). The most important effect of this change is to make uses of SMLofNJ.Cont.capture be properly tail recursive.

John Reppy

New concurrency-related instructions for x86 and amd64 code generators.

Added makefile and other support for building runtime on OpenBSD.

John Reppy

Minor fix to ml-ulex backend for regexps that match any character and perform a single action. Previously the emitted code would not allow the regexps to match any character at all.

Aaron Turon

Added build support for OpenBSD (thanks to Brian O’Hanlon).

John Reppy

Fixed Int64.fromString to use base-10.

John Reppy

Various updates to the ml-lpt tree. The documentation has been updated; bugs in the parsing of negation and character classes in ml-ulex have been fixed; and changes have been made to make the ml-lpt tools more compatible with MLton (and other SML implementations). Thanks to Matthew Fluet and Aaron Turon for their patches.

John Reppy

Implemented timer-based profiling on Windows, with behavior as close to the *nix ITIMER-based profiling as possible.

Lars Bergstrom

Finished off the Windows subset of the basis library. Added process support and various configuration and system identification utils.

Lars Bergstrom

Major changes to the RegExp library: see smlnj-lib/CHANGES for details.

John Reppy

Added the Windows Status structure

Lars Bergstrom

Added the Windows DDE structure

Lars Bergstrom

Added the Windows Config structure

Lars Bergstrom

Added outline of the Windows basis library and the basic registry functionality.

Lars Bergstrom

Fixed the amd64 code generator to compile with the current MLRISC.

Mike Rainey

nlffi was updated to work on Windows. It needed to pass in the correct value for the name of the kernel32 DLL to obtain 'base' bindings. Additionally updated the README for the most basic nlffi sample with what you need to do on MacOSX and on Windows to make it work.

Lars Bergstrom

Overwrite the SMLNJ_HOME environment variable on installation Properly change the package code so that subsequent version installations prompt for uninstall (instead of 'repair/remove')

Lars Bergstrom

Cleaned up WININSTALL file for new MSI-based setup.

Lars Bergstrom

Ensure that the size of the allocation space is at least 128K.

John Reppy

Fixed type error in ml-lpt library that occurs when compiling against a basis that was compiled with the USE_64_BIT_POSITIONS symbol set. (Thanks to Johannes Joemann)

John Reppy

Fixed the amd64 code generator to compile with the current MLRISC.

Mike Rainey

Made sml.bat more resilient to either not having run the installer and having no SMLNJ_HOME set or having just shuffled the directory around.

Lars Bergstrom

A collection of bug fixes for machine.sml in the Reactive library. (Thanks to Timothy Bourke)

John Reppy

Patches for Mac OS X 10.5 (Leopard).

John Reppy

Fixed some bugs in the AMD64 floating-point spilling code.

Mike Rainey

Added support for the atomic XCHG instruction.

Mike Rainey

Added AMD64 support for floating-point negation.

Mike Rainey

Fixed ^C handling in Windows Added a Windows installer Made it possible to build for Windows on a mapped drive from Parallels

Lars Bergstrom

Added AMD64 support for the atomic fetch and add instruction.

Mike Rainey

The GAS output now favors p2align over align, since the former is guaranteed to be consistent over multiple architectures and the latter is not.

Mike Rainey

Fixed a bug in register spilling.

Mike Rainey

Added demo support for AMD64 for quick testing and fixed support for 64-bit label constants.

Mike Rainey

Added support for the MLRISC COND instruction and the x86-64 CMOVCC instruction.

Mike Rainey

Fixed an instruction-selection bug when loading 64-bit labels.

Mike Rainey

Fixed Elaborator and Translate performance bugs

George Kuan

Fixed xorl memory argument bug.

Mike Rainey

Added f64sgn (for Real64.signBit) as a primop defined in MLRiscGen. signBit(~0.0) not handled correctly.

George Kuan

Fixed bug in {TextIO,BinIO}`.StreamIO.endOfStream` that would incorrectly signal end of stream.

John Reppy

Eliminated config/allsources. The information is now drawn directly from config/actions.

Matthias Blume

Matthias Blume

After Aaron Turon’s bug fix for ml-ulex (handling the ^ character in legacy mode), re-ran the lexer generator on all lex input files and committed the results.

This should fix the problem with ckit and nlffi that was reported by Vesa A. Norrman.

Matthias Blume

3rd merge of base from primop-branch-3 into the trunk. Additional bug fixes included:

Dave MacQueen

Fixed some bugs in new Div code in FLINT/trans/translate.sml.

Also, changed the handling of "no core access": When translate.sml needs access to a core exception at a time when the core has not been set up yet (this only happens when compiling system/smlnj/init/*), then don’t bother generating the corresponding tests.

The old scheme was to generate a bogus value to be used in place of the exception. Unfortunately, that confuses the plambda type checker. Moreover, it does not do any good, because at runtime we don’t expect such an exception to be ever raised. (The code in system/smlnj/init/* has to be written very carefully with this in mind!)

Matthias Blume

Added FSQRT instructions for the AMD64 code generator.

Mike Rainey

FLINT/trans/translate.sml now wraps all DIV/MOD/QUOT/REM operations with an explicit test for zero division. This should fixe several regressions and makes it possible for downstream optimization phases to treat these operations as "pure" when they are applied to unsigned operands.

Matthias Blume

Added the new MLRISC code generator for the AMD64. This version, in contrast to the previous one, uses SSE registers and instructions for all floating-point computations.

Mike Rainey

A number of fixes related to the formatting of dates. These include fixes for bugs #1415 and #1416. We also now correctly handle format characters that lie outside the specified set.

John Reppy

CMB (and CM) now automatically defines the CM "preprocessor" symbol NO_PLUGINS during "makeml -rebuild" or when CM operates in "slave" mode.

I changed the three locations within the sources that get compiled during CMB.make where ml-yacc or ml-lex input is processed: When NO_PLUGINS defined, then the use of the mlyacc and mllex tools is bypassed.

When bootstrapping new versions of the compiler, there can be situations where the plugin tools for ml-yacc and ml-lex (or ml-ulex) are not available or otherwise not operational. In this case one can manually define the NO_PLUGINS symbol prior to running CMB.make(). To do so, the following command should be issued at the interactive prompt:

A CMB.make with NO_PLUGINS defined relies on the existence of the files that normally would be generated by ml-yacc and ml-lex. (Copies of these files are in the repository.)

Matthias Blume

Second merge of base from primop-branch-3 into the trunk. Additional bug fixes included:

Changed the installation mechanism for CM tool plugins. These are just libaries and now get installed like ordinary libaries.

There are now a number of new installation targets that give some fine-grain control over what classes and suffixes are known, and what they will map to. See config/targets for details.

The code that caused plugin installation as part of running a tool’s "build" script has been removed. (The build script is for building, not for installing.)

Matthias Blume

Added a boolean control named cm.tolerate-tool-failures (env. variable name: CM_TOLERATE_TOOL_FAILURES). The default is false and makes CM fail if a shell tool reports a non-success exit status. If the control is set to true, then CM will press on after tool failures in the event that the target files exist (even though they are considered outdated). Turning the control to true can be useful for bootstrapping.

Matthias Blume

Merge of ml-lpt revisions for 110.64.

The name of several ml-lpt-lib modules has changed:

The ml-antlr specification format has changed: declarations such as %tokens and nonterminal definitions can occur multiple times in the same specification. The semantics are such that each new declaration extends the previous ones. This does not apply to %start or %name, of course.

Importing a grammar via %import now includes all declarations in from the imported grammar, except for %name, %entry, and %start. Tokens and nonterminals can be dropped using the new %dropping clause of the %import directive; the separate %drop and %extend have been removed.

We now allow optional type annotations on nonterminals, using the %nonterms directive as in ml-yacc.

The refcell construct is now implemented using SML’s regular reference cells, so the :== and !! notation has been deprecated.

The ml-antlr tool now does much more checking of specifications, and its error messages have been greatly improved. Error repair for generated parsers has been completely rewritten, and is now both much faster and more accurate.

ml-ulex is now more lenient with escape codes (non-SML-standard escape codes are now interpreted literally, so e.g., \| denotes “|”). Also, character classes may now include a “-” character at the beginning as is standard in most other regexp tools.

All of these changes are documented in the user guide, which has been updated and improved with this merge.

Aaron Turon

Merge of the primop3 branch (base) into the trunk to create 110.63.1. Significant changes in FLINT and the front end, mostly having to do with a reorganized system for handling primops. Various bug fixes and improvements in printing signatures.

David MacQueen

Preliminary commit of large ml-lpt revisions (more to come for 110.64). The ml-antlr specification format has changed: declarations such as %tokens and nonterminal definitions can occur multiple times in the same specification. The semantics are such that each new declaration extends the previous ones. Grammar extension constructs have also changed. We now allow type annotations on nonterminals. Finally, the refcell construct is now implemented using SML’s regular reference cells, so the :== and !! notation has been deprecated. All of these changes will appear in the 110.64 user guide for ml-lpt.

The ml-antlr tool now does much more checking of specifications, and its error messages have been greatly improved. There has also been some work on the error repair process for generated parsers, but this will be further improved in 110.64.

Aaron Turon

More Basis fixes: The Char.fromString (etc.) functions did not handle the “\uxxxx” escape sequence. There is still an outstanding bug with String.fromString the tail is a format escape. I added a comment to this effect in Basis/Implementation/string.sml. Thanks to Andreas Rossberg.

John Reppy

Added next function to Fifo and Queue modules in the SML/NJ Library.

John Reppy

More Basis fixes: Time.fmt dropped the leading “~” for negative time values that had no whole part.

John Reppy

More Basis fixes: the OS.Path module did not include the InvalidArc exception and did not do sufficient argument checking. Thanks to Stephen Weeks and Adam Chilpala.

John Reppy

Fixed an unbound functor bug in the AMD64 CM file.

Mike Rainey

Substantially changed the signature and implementation of AMD64 SVID. It now looks similar to the ia32 SVID, yet uses staged allocation.

Mike Rainey

More Basis fixes: the WORD signature was missing {to,from}`Large.` Thanks to Andreas Rossberg.

John Reppy

Fixed bogus operand sizes in AMD64 instruction spilling.

Mike Rainey

Fixed a number of inconsistencies between the Posix.TTY structure and the Basis specification. Thanks to Adam Chilpala.

John Reppy

Added preliminary support in MLRISC for Staged Allocation, a technique for specifying calling conventions (see http://www.eecs.harvard.edu/~nr/pubs/staged-abstract.html). Initially, we plan to use this code to generate C calls for the AMD64.

The staged allocation code base resides in MLRISC/staged-allocation, and specialized calling conventions go in MLRISC/ARCH/staged-allocation.

Mike Rainey

Bug fix: when SaveCState was called with two values to save, a subsequent GC could cause the RestoreCState to fail because the saved state had been promoted to tagless pair.

John Reppy

Fixed bug triggered by:

The x86MCEmitter crashed when the immediate operand to MOVB was outside of the range -128 …​ 127. I’ve changed the code so that the range check is disabled. Only the low order 8 bits of the immediate operand are now significant.

Allen Leung

Eliminated any mention of lexgen, which was an early precursor to ml-ulex.

You should update your admin directory, so the shell scripts for maintaining your local copy of the repository reflect this change.

Matthias Blume

Fixed bug in CM’s parallel make facility that failed to have the master re-link modules after letting slaves compile them.

Matthias Blume

Fixed typo in ml-build script that prevented library anchors from being registered.

Matthias Blume

Brought ml-lpt manual up-to-date with the code. Many minor improvements to the tools, and a few minor bugfixes. Prepared ml-lpt-lib for integration with new UTF8 structure. Changed the interface for creating streams in ml-ulex (we now support stream creation from several kinds of sources). Overall, the tools are now quite stable, and their interfaces are unlikely to change in a way that would break compatibility. More work, however, is needed in the documentation.

Aaron Turon

Implemented library installer. Moved CM plugin code for ml-burg, ml-lex, and ml-yacc out of CM source tree and into their respective trees. Implemented CM plugin for ml-ulex and ml-antlr. Used library installer for ml-burg, ml-ulex, and ml-antlr.

For ml-yacc and ml-lex we continue to have permanently "plugged-in" CM tools. (It turns out to be too messy to do otherwise because there is too much code that during installation relies on the presence of these tools — resulting in a tricky ordering problem.)

Matthias Blume

Added SMLofNJ.shiftArgs which is like a shell’s "shift" command. Modified CM’s startup code to use shiftArgs as it processes command line arguments. This way, the init code in each .sml-file or library that is mentioned at the top level will see only those arguments that have not yet been processed at this point. In other words, the init code can "seize control" and process the remaining command line.

Matthias Blume

Added fromList function to the ORD_SET interface and lookup to the ORD_MAP interface. See the SML/NJ Library CHANGES file for details.

John Reppy

Added the UTF8 structure and signature from the Moby compiler to the SML/NJ library (and the CML library). These modules will replace the version in the ml-lpt-lib.

John Reppy

Added entries to handle ml-lpt-lib.cm in installer.

Matthias Blume

Fixed brown-paper-bag bug with CM’s pathname handling, which made installation fail under Win32.

This supersedes the pre-brown-paper-bug release (see below).

Matthias Blume

Fixed the code in runtime/c-libs/posix-tty/{tcgetattr,tcsetattr}`.c` to get the c_cc termios data copied correctly. Also moved the allocation of the string to avoid problems if it caused a GC.

Thanks to Timothy Bourke for the bug report and fix.

John Reppy

Added code to CM’s "standard shell tool" implementation which causes it to tolerate (with a warning) the situation where the shell command fails (e.g., due to the shell command’s non-existence) as long as all target files exist.

This makes it possible to, e.g., build ml-yacc from sources even if svn checkout messed up the time stamps on files in such a way that yacc.grm is younger than yacc.grm.sml or yacc.grm.sig. (Ml-yacc would be needed to re-process yacc.grm, but obviously it might not yet be available at that time.)

Matthias Blume

Fixed the types of recvVecFrom, recvVecFrom', recvVecFromNB, and recvVecFromNB' in the SOCKET signature. This error is actually in the SML Basis specification too.

John Reppy

CM now reports undefined anchors as errors and aborts execution rather than silently pressing on using bogus values.

Matthias Blume

Use Say.vsay for printing the “[autoloading]” message, so #set CM.Control.verbose false (or -Ccm.verbose=false) can be used to suppress them.

Matthias Blume

Fixed bug in CM where “with:” specifications that affect compilation (as opposed to parsing) were ignored.

Matthias Blume

This is the first subversion-hosted release. There are also changes to the directory layout. Here is a table:

(All pathnames are relative to the SML/NJ "root" directory.)

In addition, there is also a new

directory containing two new program generator tools: ml-ulex and ml-antlr (a lexer generator that handles unicode and an ANTRL-inspired LL(k) parser generator). These tools are currently "beta-quality"

The latest versios of the sources can now be obtained anonymously via subversion. For this, it is useful to first check out

and put the resulting directory on your shell’s PATH. This provides access to three shell scripts: checkout-all.sh, stat-all.sh, and refresh-all.sh.

To create a freshly checked-out copy of the sources, do

where dir is the optional SML/NJ root directory (default is ".").

This creates the above directory layout. Each subdirectory of the root is under individual subversion control. The stat-all.sh and refresh-al.sh scripts apply "svn stat" or "svn update" to each of these subtrees.

Matthias Blume

Reorganized directory layout.

This is a temporary solution, more reorganization is to come.

The basic idea is to have a number of toplevel trees, each corresponding to a well-defined part of the overall system. Each part can be maintained individually, even in separate source repositories, although currently we still serve everything out of the main smlnj-gforge tree.

Installer and and scripts have been updated to reflect the new layout. The installer (base/src/system/smlnj/installer) is now "scriptable" to avoid burning too much knowledge about the layout into SML source code. The main script used by the installer is in config/actions.

The main change is that many of the subdirectories of what used to be known as "src" have moved to the toplevel. The "src" directory itself has moved down into a subtree called "base". (We may eventually get rid of the extra level of indirection represented by "src".)

The layout is now as follows:

In $gf/admin there are a few useful shell scripts for checking out and maintaining the entire collection of trees:

Matthias Blume

Merged code for AMD64 backend (Mike Rainey’s work). Everything is hooked up but untested.

Matthias Blume

I am freezing 110.59. Changes other than the version-number increase:

Matthias Blume

Implemented ml-makedepend (i.e., CM.sources) in a better (more robust) way. This should hopefully fix the ml-makedepend problem permanently.

Matthias Blume

Fixed long-standing bug with ml-makedepend where it would output a spurious dependency to a non-existing file. (This is a simple fix. It might need further looking into.)

Matthias Blume

Committed patches received from Johannes 5 Joemann (joemann@befree.free.de) that enable heap2exec under Linux and FreeBSD.

Matthias Blume

MLRISC changes:

John Reppy

Incorporated several bugfixes to lexgen. Compiler now compiles to fixpoint when using lexgen instead of ml-lex.

Matthias Blume

Removed ml-flex and added lexgen instead, using Aaron Turon’s newly provided tarball. The generated lexers still have problems.

Matthias Blume

Removed ml-flex's dependency on regexp-lib.cm. Turned installation of ml-flex on by default.

Matthias Blume

Added ml-flex sources. Partially integrated, but should not be turned on yet! (Read: leave it commented-out in config/targets!)

Matthias Blume

Changes to support building on x86-64 systems (using the 32-bit mode). Also cleaned up signal handling on Linux. Support for pre-2.2 Linux kernels dropped.

John Reppy

CM has changed. Updated the script for rebuilding the MLRISC generated files.

Allen Leung

Hooked code for Darwin-specific Intel ABI into main compiler. (This is a temporary solution which relies on the fact that the compiler itself does not use NLFFI. Eventually we need to divorce intel mac from generic x86 unix code and make separate sets of binaries.)

Matthias Blume

Changed MLRISC x86 CCalls for partial support of Mac OS X ABI.

John Reppy

Changes to support Mac OS X on Intel hardware. The C-calls support in MLRISC must also be updated to support the Mac OS X ABI.

John Reppy

Improved error reporting and handling in CM.

Matthias Blume

Enabling $/html-lib.cm in config/preloads did not work. This is fixed now. (Since the anchor mapping for html-lib.cm is not yet in effect at the time when config/preloads is processed, the library has to be referred to by another name. In 110.57 this would be $SMLNJ-LIB/HTML/html-lib.cm. I arranged for $smlnj/smlnj-lib/html-lib.cm to be valid as well.)

Thanks to Todd Wilson (Fresno) for alerting me to this issue.

Matthias Blume

Fixed a problem in config/install.sh where it tries to "fish" the name of the CM metadata directory from the wrong place (because the physical location of basis.cm has changed). Also, corrected the path anchor for $/html-lib.cm. (Thanks to M. Fluet for pointing out these problems.)

Matthias Blume

Fixed problem with bogus exception message when using back-trace facility.

Matthias Blume

Added simple implementation of Array2.copy. (Warning: mostly untested.)

Matthias Blume

Reversed change to src/system/smlnj/internals/versiontool.cm. This file gets loaded as a tool — by the equivalent of CM.make during the run of CMB.make. Thus, CMB’s path configuration is meaningless for it. Instead, it has the status of "user code", so it should use $/basis.cm to refer to the Basis library. (At least that’s true for the purpose of bootstrapping the previous change. In the future it might make sense to have versiontool.cm refer to $smlnj/basis/basis.cm, i.e., the version of the Basis that the compiler itself uses.)

Also patched src/system/testml to have it activate those extra anchor bindings in config/extrapathconfig.

Matthias Blume

This change affects the way the following libraries are tied into the system:

These libraries are now internally (as seen from the source code of the implementation itself) known by the following names:

This makes it possible to work with code that requires different versions of these libraries, and which refers to these libraries using their "default" names (i.e., the first set of names above). In other words, one can un-define or re-define those default names without compromising the proper functioning of the compiler itself.

A similar procedure had already been performed for several of the MLRISC libraries that are linked into the compiler. I did some cleanup on this code.

A new file in the config directory (named extrapathconfig) is responsible for setting up path anchors that the compiler itself does not need, but that are typically required by user code.

Matthias Blume

Fixed erroneous out-of-bounds test in the “update” function of various *ArraySlice modules. (Thanks to Vesa A. Norrman for pointing out the problem.)

Pushed some Basis changes through ML-Lex, CML, and eXene.

Matthias Blume

Fixed a Basis incompatibility: The depreciated function Substring.all was removed (use Substring.full instead).

John Reppy

Tweaked interval set API in SML/NJ library; see the CHANGES file for details.

John Reppy

Matthias Blume

Fixed a Basis incompatibility: Pack<N>{Big,Little} structures should be named PackWord<N>{Big,Little}.

John Reppy

Fixed a minor Basis incompatibility: hex digits should be upper case.

John Reppy

Added interval sets to utility library (signatures INTERVAL_DOMAIN and INTERVAL_SET, and functor IntervalSetFn).

John Reppy

Add Zhong Shao’s fix for datatype equality functions.

John Reppy

Bug fix a bug found by Carl Hauser. There was a typo in the reload code for FCMP in x86SpillInstr.sml.

Allen Leung

Removed some debugging code in file x86Asm.sml. The function emit_operand was printing out debugging output.

Allen Leung

Fixed ml-lex to recognize “\r” as representing carriage return.

John Reppy

Fixed ml-yacc to work on files with non-native end-of-line encodings (e.g., Windows text file on a Unix system).

John Reppy

Added changes from Dominic Evans (oldmanuk (at) gmail (dot) com) to support HPUX 11.

John Reppy

Changes to the SML/NJ Library. See smlnj-lib/CHANGES for details.

John Reppy

Fixed reversed logic for deciding whether to "copy up" or "copy down" in *-array-slice.sml.

Matthias Blume

A typo in the Cygwin code fixed.

Allen Leung

Updated Cygwin’s fault/signal handling to match the Windows version. Updated the export list.

Allen Leung

Added support scripts for Mac OS X PackageMaker and modified config/install.sh so that it supports re-dumping a heap image after customization.

Matthias Blume

Un-overloaded / to work around bug in overloading resolution code.

Matthias Blume

Added mechanism for re-creating a heap file for the interactive system after configuration variables have been changed.

This is much like SMLofNJ.exportML, but starting from the resulting heap does not return to the caller of CM.redump_heap but restarts the interactive system from scratch. The original call of CM.redump_heap does not return but ends the interactive session. Thus, CM.redump_heap is a lot like SMLofNJ.exportFn.

Internally, redump_heap winds the dynamic execution context back to the point where the original heap image was created and re-executes the heap image generation code in the boot code.

Matthias Blume

Added a hack to the existing hack known as Word64 to make fromString behave correctly. I am still not sure whether Word64.scan will work as specified with respect to the interaction of radix and prefix.

Matthias Blume

Added a gc protocol checking phase. This phase is enabled with the flag "check-gc". "debug-check-gc" turns on the verbose mode.

Allen Leung

Fixed a bug in the implementation of div and mod for IntInf. Thanks to Neophytos Michael for reporting the problem.

Matthias Blume

Added the join combinator to the ParserComb module in the SML/NJ Library.

Matthias Blume

Fixed serious bug (brown paper bag variety) in new implementation of structure Atom in CML. (I had accidentally used a mailbox instead of an mvar, leaving the door open for races.)

Matthias Blume

Brought back SMLofNJ.Susp. The underlying suspension type is the one implemented in Core, which means that it is the same as the one used by the lazy extension.

Matthias Blume

Simpler and at the same time more general implementation of structure Atom in CML.

Matthias Blume

Created new “tools” directory under “src” and moved “TraceDebugProf” there.

Matthias Blume

Implemented “long long” arguments and results for NLFFI. (Only the PPC/MacOS implementation is complete, the other backends still need to be updated.)

Matthias Blume

Minor cleanup in ML-Yacc rule printing mechanism. This should fix a problem with certain "as" patterns which previously got rendered using incorrect syntax.

Matthias Blume

Made time profiling code (interrupt handler) in runtime system aware of new array representation.

Matthias Blume

Implemented new (but still experimental) heap2exec facility. This is tested under Mac OS X and should work under Linux (will test shortly). It will probably also work on the Sparc (will test some time later). Also removed old “HACKED_STANDALONE” hack from runtime

To be able to test heap2exec, uncomment the request for “heap2asm” in config/targets prior to installation. (Notice that this is different from "heap2exec" mentioned below. Not a typo.)

To perform an actual test, run the command

(You can put heap2exec on your shell’s path.)

For example, run

This will create a standalone executable called “mly” that you can then invoke directly as a command.

Matthias Blume

fixed off-by-one error in ML_STRING macro (globals.c)

Matthias Blume

Made ml-build script "smarter" (but only very little).

Matthias Blume

Some cleanup in the $c/memory.cm library: separated some concerns by moving allocation code and memory access code each into their own files.

Matthias Blume

The Unix I/O library of SML/NJ on Cygwin does not understand Windows style pathname, so problems arise when SMLNJ_HOME is set to a Windows style pathname. The _run-sml script now converts SMLNJ_HOME to a POSIX pathname on Cygwin.

Allen Leung

Last-minute changes incorporated into 110.52. Release tag moved.

The changes: - HashString.hashString' → HashString.hashSubstring - bug fix in UnivariateStats

+ Matthias Blume

More on the space problem (this time for Win32).

Matthias Blume

Hacked some of the scripts (in particular: the installer) to cope with spaces in filenames a bit better. But beware: the current "solution" is likely still full of bugs and inherently incomplete. (We need to do away with those shell scripts for a comprehensive solution.)

Matthias Blume

Fixed bug in code for ml-makedepend.

Matthias Blume

Added two simple but potentially useful statistics modules to SML/NJ Library. (See CHANGES file there.)

Matthias Blume

Updates to SML/NJ Library

(Also see CHANGES file for smlnj-lib.)

Matthias Blume

Made sure CML compiles when the Position structure is Int64.

Matthias Blume

The compiler can now be compiled in a mode that makes structure Position equal to Int64. The default, however, is unchanged (Position is Int31) for the time being.

To enable 64-bit positions, use the following procedure:

This is barely tested. The only test so far was a little SML program counting the number of characters in an 8-gigabyte file by reading it character-by-character. That test was successful.

In support of 64-bit positions, a number of new functions have been added to the runtime system.

Matthias Blume

Fixed a problem with unhelpful error messages related to problems with .cm or .sml files that appear as part of the sml command line.

Matthias Blume

Enabled dlopen and friends for FreeBSD (as recommended by Johannes 5 Joemann).

Matthias Blume

Added support for MLTree constructs LIVE and KILL to all the architectures.

Allen Leung

Structure Int64 fully hooked in. (The implementation is not very efficient, though.)

Matthias Blume

All the pieces of Word64 are now there, with the exception of the conversions from and to LargeWord. (Eventually these need to be identities, but for the time being they don’t even make sense because LargeWord is 32-bit wide.)

Also started to add similar support for Int64, but major pieces of that are still missing.

Matthias Blume

Structure Word64 is now (almost) complete, word literals and patterns seem to work. There are a few odd pieces missing. In particular, I didn’t do the {from,to}`LargeWord` parts because LargeWord is still Word32 at the moment.

Making Word64 official would mean that LargeWord becomes Word64. But this requires extreme care because most word-word conversions have to go through LargeWord, so making a mistake means loss of efficiency or worse. Eventually there will be a solution similar to (but actually simpler than) what I did with IntInf.

Matthias Blume

More 64-bit hacking (but still not even half-way there yet). Also, some assorted improvements to the handling of 8-bit words.

Matthias Blume

Started adding some infrastructure for supporting 64-bit int- and word-types. (Still in its very early stages.)

Matthias Blume

(As a result, a version change can be done by just editing config/version, the rest is now automatic.)

Matthias Blume

BackTrace.monitor now also reports the source of the exception that triggered the trace.

Matthias Blume

This is the HISTORY entry for two earlier commits, both concerning the x86 c-calls code in MLRISC:

Matthias Blume

John discovered a bug in the syntax of fucomip. The opcodes FU?COMIP? have been changed to

Allen Leung

Added a mechanism for getting back-trace information from standalone programs. Here is how it works:

I changed the library name $/trace-debug-profile.cm to $smlnj-tdp/plugins.cm, and added the following new libraries:

Matthias Blume

Added an "obsolete" warning for the "group owner" syntax to CM's parser.

Eliminated group owner specs from .cm files throughout the source tree.

Matthias Blume

To get the benefits of any of these plugin modules, the code in question must be compiled with TDP instrumentation turned on. This can be done by setting SMLofNJ.Internals.TDP.mode to true. (The ref cell is also controlled via the -Ctdp.instrument=…​ switch.)

Plugins are selected at link time. (Pre-compiled instrumented code can be re-loaded with different plugins in effect.) When an instrumented module is linked, whatever plugins are at that time enabled will come into effect for that module.

To enable the back-trace plugin, load library $/trace-debug-profile.cm and invoke BackTrace.install() (e.g., from the interactive prompt). To enable the coverage plugin, load the same library and invoke Coverage.install().

Back-traces are generated automatically on uncaught exceptions and when the code in question explicitly invokes BackTrace.trigger().

Coverage (and execution frequency-) information must be queried explicitly by calling Coverage.not_covered and Coverage.hot_spots.

Matthias Blume

Snapshot of a significant overhaul of how the trace/debug/profile support is hooked into the system (specifically: Core and SMLofNJ.Internals).

Matthias Blume

Some rationalization of names:

This is is preparation of using the original back-trace instrumentation for other purposes. "TDP" stands for Trace/Debug/Profile.

The control flag controlling whether instrumentation is on or off is now registered under a different name, so instead of running sml as

one has to say

Matthias Blume

Made some minor modifications to elabcore.sml to have source regions be propagated more tightly — resulting in better (i.e., smaller) regions being reported in error- and debug messages.

Matthias Blume

Fixed handling of keywords in .cm files: After seeing "is" the lexer treats subsequent occurrences of "group", "library", "source", "is", "*", and "-" as ordinary identifiers rather than keywords.

Most seriously, this fixes a problem with CM’s "shell" tool. The tool is supposed to accept a tool argument called "source", but this did not work because of the clash with the keyword.

Matthias Blume

Assorted cleanup work:

Matthias Blume

Cleaned up the absyn to reflect the invariant that HANDLE always carries a FNexp as part of the type definition. This eliminates some superfluous sanity checks at runtime down the road.

Some minor cleanup of the btrace code.

Matthias Blume

Added hack to make slave mode work in the presence of the version tool. (Still, since the master does two passes over the code for CMB.make, the release number gets bumped twice when slaves are attached. I don’t know if this is worth fixing…​)

Matthias Blume

Implemented some CM magic to have file src/compiler/TopLevel/main/version.sml generated automagically. The version is taken from two files: config/version and config/release. The first is expected to contain a two-part version number such as 110.49. The second should contain a single number, but it may be missing.

If the environment variable VERSIONTOOL_BUMP_RELEASE is defined at the time the version tool is loaded (which is the first time you say CMB.make), then the tool will increment the value stored in config/release every time CMB.make is invoked.

The binfile format is now insensitive to anything beyond the first two components of a version number, so bumping the release does not render binfiles incompatible. Auto-bumping can be used to keep track of versions during development without invalidating existing binfiles.

In any case, every CMB.make updates the date information in version.sml. (This is the date that is printed in the banner.)

Matthias Blume

Some cleanup of the controls code.

Matthias Blume

Added two pieces of functionality to the Controls interface:

grabs the current value of the control in stage 1 and restores it in stage 2.

stores the given value into the control in stage 2 (i.e., delayed) but does all error checking in stage 1. (This is for string controls that need to do parse their argument — something that might fail. In some cases, notably in CM, one already knows the intended argument but wants to delay the actual assignment until a time when error recovery would be more difficult.)

Also changed the handling of controls in tool arguments to classes “sml” and “lazysml”:

Matthias Blume

Added a new tool class called “lazysml” to CM’s tool chest. The only difference to “sml” is that compilation is done with Control.lazysml set to true. A source of class “lazysml” is automatically recognized by a file name suffix of “.lml”.

In addition to the above feature, the original class “sml” now also supports a tool argument “lazy” which has the same effect. As a result, the following three lines are equivalent:

The setting goes into effect both during parsing and during compilation. The original setting is restored right after parsing and after compilation, respectively.

In addition to all the above, there is also a general mechanism to set ANY of the "controls" that are available at the command line via “-C…​” on a per-sml-file basis. The same rules that apply for “lazy” apply as well. (In fact, “lazy” is implemented as a special case of the general mechanism.)

The .cm file syntax uses a new keyword tool argument called “with”. There are several ways of indicating the desired settings: