The main thrust behind this API-breaking release is to be able to support a protocol for Graphics3D.

It supports a new Graphics3D protocol is currently expressed in JSON. There is an independent threejs-based module to implement this. Tiago Cavalcante Trindade is responsible for this code.

The other main API-breaking change is more decentralization of the Mathics Documentation. A lot more work needs to go on here, and so there will be one or two more API breaking releases. After this release, the documentation code will be split off into its own git repository.

Enhancements

• a Graphics3D protocol, mentioned above, has been started
• mathics.setting has been gone over to simplify.
• A rudimentary and crude SVG Density Plot was added. The prior method relied on mysterious secret handshakes in JSON between Mathics Core and Mathics Django. While the density plot output was nicer in Mathics Django, from an overall API perspective this was untenable. A future version may improve SVG handling of Density plots using elliptic density gratings in SVG. And/or we may define this in the JSON API.
• SVG and Asymptote drawing now includes inline comments indicating which Box Structures are being implemented in code
• Elliptic arcs are now supported in Asymptote. There still is a bug however in calculating the bounding box when this happens.

Documentation

• Document data used in producing PDFs and HTML-rendered documents is now stored in both the user space, where it can be extended, and in the package install space -- which is useful when there is no user-space data.
• The documentation pipeline has been gone over. Turning the internal data into a LaTeX file is now a separate own program. See mathics/doc/test/README.rst for an overview of the dataflow needed to create a PDF.
• Summary text for various built-in functions has been started. Thes summaries are visible in Mathics Django when lists links are given in Chapters, Guide Sections, or Sections.
• A Sections for Lists has been started and grouping for these have been added. So code and sections have moved around here.
• Regexp detection of tests versus document text has been improved.
• Documentation improved
• The flakiness around showing sine graphs with filling on the axes or below has been addressed. We now warn when a version of Asymptote or Ghostscript is used that is likely to give a problem.

Bugs

• A small SVGTransform bug was fixed. Thanks to axelclk for spotting.
• A bug in image decoding introduced in 3.1.0 or so was fixed.
• A bug SVG LineBoxes was fixed

Regressions

• Some of the test output for buitins inside a guide sections is not automatically rendered
• Density plot rendered in Mathics Django do not render as nice since we no longer use the secret protocol handshake hack. We may fix this in a future release
• Some of the Asymptote graphs look different. Graphic3D mesh lines are not as prominent or don't appear. This is due to using a newer version of Asymptote, and we will address this in a future release.

New variables and builtins

• Arrow for Graphics3D (preliminary)
• Cylinder (preliminary)
• Factorial2 PR #1459 Issue #682

Enhancements

Large sections like the "Strings and Characters", "Integer Functions" and "Lists" sections have been broken up into subsections. These more closely match online WL "Guide" sections. This is beneficial not just in the documentation, but also for code organization. See PRs #1464, #1473.

A lot more work is needed here.

The Introduction section of the manual has been revised. Licensing and Copyright/left sections have been reformatted for non-fixed-width displays. #1474

PolarPlot documentation was improved. #1475.

A getter/setter method for Mathics settings was added #1472.

Bugs

• Add requirements-*.txtto distribution files. pip install Mathics3[dev] should work now. PR #1461
• Some PointBox bugs were fixed
• Some Arrow3DBox and Point3DBox bugs were fixed PR #1463
• Fix bug in mathics CLI when -script and -e were combined PR #1455

CHANGES

3.0.0

Overall there is a major refactoring underway of how formatting works and its interaction with graphics. More work will come in later releases.

Some of the improvements are visible not here but in the front-ends mathicsscript and mathics-django. In mathicsscript, we can now show SVG images (via matplotlib). In Mathics Django, images and threejs graphs are no longer embedded in MathML.

A lot of the improvements in this release were done or made possible with the help of Tiago Cavalcante Trindade.

Enhancements

It is now possible to get back SVG, and graphics that are not embedded in MathML.

The code is now Pyston 2.2 compatible. However scipy lxml are not currently available on Pyston so there is a slight loss of functionality. The code runs about 30% faster under Pyston 2.2. Note that the code also works under PyPy 3.7.

Bugs

• Tick marks and the placement of numbers on charts have been corrected. PR #1437
• Asymptote now respects the PointSize setting.
• In graphs rendered in SVG, the PointSize has been made more closely match Mathematica.
• Polygons rendered in Asymptote now respects the even/odd rule for filling areas.

Density Plots rendered in SVG broke with this release. They will be reinstated in the future.

Documentation

Go over settings file to ensure usage names are full sentences.

We have started to put more builtins in the sections or subsections following the organization in Mathematics 5 or as found in the online Wolfram Language Reference. As a result, long lists in previous topics are a bit shorter and there are now more sections. This work was started in 2.2.0.

More work is needed on formatting and showing this information, with the additional breakout we now have subsections. More reorganization and sectioning is needed.

These cleanups will happen in a future version.

Chapters without introductory text like Structural Operations, or Tensors have had descriptions added.

Sections that were empty have either been expanded or removed because the underlying name was never a user-level built in, e.g. the various internal Boxing functions like DiskBox, or CompiledCodeBox

Documentation specific builtins like PolarPlot or BernsteinBasis have been added improved, and document examples have been revised such as for PieChart, Pi and others.

The Mathics Gallery examples have been updated.

Some slight improvements were made to producing the PDF and more kinds of non-ASCII symbols are tolerated. Expect more work on this in the future via tables from the Mathics Scanner project.

Chapters are no longer in Roman Numerals.

Internal changes

• docpipline.py accepts the option --chapters or -c to narrow tests to a particular chapter
• Format routines have been isolated into its own module. Currently we have format routines for SVG, JSON and Asymptote. Expect more reorganization in the future.
• Boxing routines have been isolated to its own module.
• The entire code base has been run through the Python formatter black.
• More Python3 types to function signatures have been added.
• More document tests that were not user-visible have been moved to unit tests which run faster. More work is needed here.

Package update

• SymPy 1.8

New variables and builtins

• Arg
• CoefficientArrays and Collect (#1174, #1194)
• Dispatch
• FullSimplify
• LetterNumber #1298. The alphabet parameter supports only a minimal number of languages.
• MemoryAvailable
• MemoryInUse
• Nand and Nor logical functions.
• Series, O and SeriesData
• StringReverse
• $SystemMemory
• Add all of the named colors, e.g. Brown or LighterMagenta.

Enhancements

• a function evaluate_predicate allows for a basic predicate evaluation using $Assumptions.
• Attributes accepts a string parameter.
• Cases accepts Heads option. Issue #1302.
• ColorNegate for colors is supported.
• D and Derivative improvements.
• Expand and ExpandAll now support a second parameter patt Issue #1301.
• Expand and ExpandAll works with hyperbolic functions (Sinh, Cosh, Tanh, Coth).
• FileNames returns a sorted list (#1250).
• FindRoot now accepts several optional parameters like Method and MaxIterations. See Issue #1235.
• FixedPoint now supports the SameTest option.
• mathics CLI now uses its own Mathics settings.m file
• Prepend works with DownValues Issue #1251
• Prime and PrimePi now accept a list parameter and have the NumericFunction attribute.
• Read with Hold[Expression] now supported. (#1242)
• ReplaceRepeated and FixedPoint now supports the MaxIteration option. See Issue #1260.
• Simplify performs a more sophisticated set of simplifications.
• Simplify accepts a second parameter that temporarily overwrites $Assumptions.
• StringTake now accepts form containing a list of strings and specification. See Issue #1297.
• Table [expr, n] is supported.
• ToExpression handles multi-line string input.
• ToString accepts an optional form parameter.
• ToExpression handles multi-line string input.
• $VersionNumber now set to 10.0 (was 6.0).
• The implementation of Streams was redone.
• Function mathics.core.definitions.autoload_files was added and exposed to allow front-ends to provide their own custom Mathics. settings.
• String output in the mathics terminal has surrounding quotes to make it more visually distinct from unexpanded and symbol output. To disable this behavior use --strict-wl-output.

Bug fixes

• SetTagDelayed now does not evaluate the RHS before assignment.
• $InstallationDirectory starts out Unprotected.
• FindRoot now handles equations.
• Malformed Patterns are detected and an error message is given for them.
• Functions gone over to ensure the Listable and NumericFunction properties are correct.

Incompatible changes

• System`$UseSansSerif moved from core and is sent front-ends using Settings`$UseSansSerif.

Internal changes

• doctest accepts the option -d to show how long it takes to parse, evaluate and compare each individual test. -x option (akin to pytests -x is a short-hand for stop on first error
• Some builtin functions have been grouped together in a module underneath the top-level builtin directory. As a result, in the documents you will list some builtins listed under an overarching categery like Specific Functions or Graphics, Drawing, and Images. More work is expected in the future to improve document sectioning.

New builtins

• ArcTanh
• ByteArray
• CreateFile
• CreateTemporary
• FileNames
• NIntegrate
• PartitionsP
• $Notebooks

Enhancements

• The Mathics version is checked for builtin modules at load time. A message is given when a builtin doesn't load.
• Automatic detection for the best strategy to numeric evaluation of constants.
• FileNameJoin now implements OperatingSystem option
• Mathics functions are accepted by Compile[]. The return value or type will be Compile[] and CompiledFunction[]. Every Mathics Expression can have a compiled form, which may be implemented as a Python function.
• Equal[] now compares complex against other numbers properly.
• Improvements in handling products with infinite factors: 0 Infinity-> Indeterminate, and expr Infinity-> DirectedInfinite[expr]
• $Path is now Unprotected by default
• Read[] handles expressions better.
• StringSplit[] now accepts a list in the first argument.
• SetDelayed[] now accepts several conditions imposed both at LHS as well as RHS.
• Axes for 2D Plots are now rendered for SVGs
• InsertBox accepts an opaque parameter

=========

• TeXForm[] for integrals are now properly formatted.

Pymathics Modules

• Pymathics modules now can run initialization code when are loaded.
• The builtins list is not hardliked to the library anymore. This simplifies the loading and reloading of pymathics modules.
• Decoupling of BoxConstructors from the library. Now are defined at the level of the definition objects. This is useful for customizing the Graphics output if it is available.

Miscellanea

• A pass was made to improve Microsoft Windows compatability and testing Windows under MSYS.
• Include numpy version in version string. Show in CLI
• Small CLI tweaks --colors=None added to match mathicsscript.
• In the BaseExpression and derivated classes, the method boxes_to_xml now are called boxes_to_mathml.
• In the format method of the class Evaluation, the builtin ToString is called instead of boxes_to_text
• In order to control the final form of boxes from the user space in specific symbols and contexts.
• GraphicsBox now has two methods: to_svg and to_mathml. The first produces SVG plain text while the second produces <mglyph ...> tags with base64 encoded SVGs.

What's to expect in a Future Release

• Improved Equal See PR #1209
• Better Unicode support, especially for Mathics operators
• Improved D[] and Derivative[] See PR #1220.
• Improved performance
• Collect[] See Issue #1194.
• Series[] See Issue #1193.

In order to accommodate growth and increase use pieces of Mathics inside other packages, some parts of Mathics have been split off and moved to separately installable packages. In particular:

• The Django front-end is now a PyPI installable package called Mathics-Django.
• Scanner routines and character translations tables to/from unicode and noting some character properties.
• Specific builtins involving heavy, non-standard routines were moved to pymathics modules.

New builtins

• AnglePath, AnglePathFold, AngleVector
• BoxData, TextData, InterpretationBox, StyleBox, TagBox, TemplateBox, ButtonBox, InterpretationBox
• ContinuedFraction
• ConvertCommonDumpRemoveLinearSyntax and SystemConvertersDump` context variables
• FirstCase, Lookup, Key, Lookup and Failure
• Haversine, InverseHaversine
• Insert and Delete
• LerchPhi
• MathicsVersion (this is not in WL)
• NumberQ
• PossibleZeroQ PR #1100
• Run
• Show
• SympyObject
• TimeRemaining and TimeConstrained
• \[RadicalBox]
• Improving support for options in the Plot module: Axes, Filling, ImageSize, Joined

New Constants

Mathematical Constants is now its own module/section. Constants have been filled out. These constants have been added:

• Catalan
• Degree
• Glaisher
• GoldenRatio
• Khinchin

Many of these and the existing constants are computable via mpmath, NumPy, or Sympy.

Settings through WL variables

Certain aspects of the configuration of the kernel are now controlled by variables, defined in /autoad/settings.m.

• $GetTrace (False by default). Defines if when a WL module is load through Get, definitions will be traced (for debug).
• $PreferredBackendMethod Set this do whether to use mpmath, NumPy or Sympy for numeric and symbolic constants and methods when there is a choice ("sympy" by default) (see #1124)

Enhancements

• Add Method option "mpmath" to compute Eigenvalues using mpmath (#1115).
• Improving the support for OptionValue and OptionsPattern (#1113)

Bug Fixes

There have been numerous bug is driven by working on Combinatorica V0.9 and CellsToTeX.

• Sum involving numeric integer bounds involving Mathics functions fixed.
• Equal UnEqual testing on Strings (#1128).

Document updates

• Start a readthedocs Developers Guide

Enhancements and Bug fixes:

• Fix evaluation timeouts
• Sum's lower and upper bounds values can now be Mathics expressions

Miscellanea

• Enlarge of the set of gries_schneider tests
• Improvement in the way builtins modules are loaded at initialization time (#1138).

In order to accommodate growth and increase use pieces of Mathics inside other packages, some parts of Mathics have been split off and moved to separately installable packages. In particular:

• The Django front-end is now a PyPI installable package called Mathics-Django.
• Scanner routines and character translations tables to/from unicode and noting some character properties.
• Specific builtins involving heavy, non-standard routines were moved to pymathics modules.

New builtins

• AnglePath, AnglePathFold, AngleVector
• BoxData, TextData, InterpretationBox, StyleBox, TagBox, TemplateBox, ButtonBox, InterpretationBox
• ContinuedFraction
• ConvertCommonDumpRemoveLinearSyntax and SystemConvertersDump` context variables
• FirstCase, Lookup, Key, Lookup and Failure
• Haversine, InverseHaversine
• Insert and Delete
• LerchPhi
• MathicsVersion (this is not in WL)
• NumberQ
• PossibleZeroQ PR #1100
• Run
• Show
• SympyObject
• TimeRemaining and TimeConstrained
• \[RadicalBox]
• Improving support for options in the Plot module: Axes, Filling, ImageSize, Joined

New Constants

Mathematical Constants is now its own module/section. Constants have been filled out. These constants have been added:

• Catalan
• Degree
• Glaisher
• GoldenRatio
• Khinchin

Many of these and the existing constants are computable via mpmath, NumPy, or Sympy.

Settings through WL variables

Certain aspects of the configuration of the kernel are now controlled by variables, defined in /autoad/settings.m.

• $GetTrace (False by default). Defines if when a WL module is load through Get, definitions will be traced (for debug).
• $PreferredBackendMethod Set this do whether to use mpmath, NumPy or Sympy for numeric and symbolic constants and methods when there is a choice ("sympy" by default) (see #1124)

Enhancements

• Add Method option "mpmath" to compute Eigenvalues using mpmath (#1115).
• Improving the support for OptionValue and OptionsPattern (#1113)

Bug Fixes

There have been numerous bug is driven by working on Combinatorica V0.9 and CellsToTeX.

• Sum involving numeric integer bounds involving Mathics functions fixed.
• Equal UnEqual testing on Strings (#1128).

Document updates

• Start a readthedocs Developers Guide

Enhancements and Bug fixes:

• Fix evaluation timeouts
• Sum's lower and upper bounds values can now be Mathics expressions

Miscellanea

• Enlarge of the set of gries_schneider tests
• Improvement in the way builtins modules are loaded at initialization time (#1138).

This may be the last update before some major refactoring and interface changing occurs.

In a future 2.0.0 release, Django will no longer be bundled here. See mathics-django for the unbundled replacement.

Some changes were made to support Pymathics Graph, a new graph package bundled separately, and to support the ability for front-ends to handle rendering on their own. Note that currently this doesn't integrate well nto the Django interface, although it works well in mathicsscript.

Package Updates

• sympy 1.7.1

Mathics Packages added:

• DiscreteMath`CombinatoricaV0.9 (preferred) aand DiscreteMath`CombinatoricaV0.6.

Both of these correspond to Steven Skiena's older book: "Implementing Discrete Mathematics: Combinatorics and Graph Theory" book.

If you have a package that you would like included in the distribution, and it works with Mathics, please contact us.

Rubi may appear in a future release, possibly in a year or so. However this might be speeded up if we can get people to help out with this.

New builtins:

• StirlingS1, StirlingS2 (not all WL variations handled)

• MapAt (not all WL variations handled)

• PythonForm, SympyForm: these is not in WL. It simply will show a crude translation to sympy or python. Expect more and better translation later

• Throw and Catch

• With

• FileNameTake

Enhancements and Bug fixes:

---

• Workaround for Compile so it accepts functions ##1026
• Add Trace option to Get. Get["fn", Trace->True] will show lines as they are read.
• Convert to/from Boolean types properly in from_python, to_python. Previously they were 0, and 1.
• Extend DeleteCases to accept a levelspec parameter.
• Set Evaluation#exc_result to capture Aborted, Timeout, Overflow1`, etc.
• ImageData changed to get bits {0,1}, not booleans as previously.
• Add tokenizer symbols for <-> and -> and the unicode versions of those.
• Small corrections to Needs, e.g check if already loaded, correct a typo, etc.
• System`$InputFileName is now set inside Needs and Get
• Install shell scripts dmathicserver, dmathicsscript, and dmathics to simplify running docker
• Adjust $InputFileName inside Get and Needs.

Incompatibilities

So we can get onto PyPI, the PyPI install name has changed from Mathics to Mathics3.

Enhancements and Bug fixes:

• Add Symbolic Comparisons. PR #1000
• Support for externally PyPI-packagable builtin modules - PyMathics.
• SetDirectory fixes. PR #994
• Catch `PatternError Exceptions.
• Fix formatting of .. and ... (RepeatAll)
• tokenization of \. without a following space (ReplaceAll). Issue #992
• Support for assignments to named `Pattern`
• Improving support for `Names. PR #1003
• Adding a MathicsSession class to simplify running Mathics from Python. PR #1001
• Improving support for `Protect` and `Unprotect` list of symbols and regular expressions. PR #1003

Package Updates

All major packages that Mathics needs have been updated for more recent releases. Specifically These include:

• Python: Python 3.6-3.9 are now supported.
• cython >= 0.15.1
• Django 3.1.x
• mpmath >= 1.1.0
• sympy 1.6.2

New features (50+ builtins):

• Association, AssociationQ, FirstPostion, LeafCount
• Association, AssociationQ, Keys, Values #705
• BarChart[], PieChart, Histogram, DensityPlot #499
• BooleanQ, DigitQ and LetterQ
• CharacterEncoding option for Import[]
• Coefficient[], Coefficient[x * y, z, 0], Coefficient*[],
• DiscreteLimit #922
• Environment
• File read operations from URLs
• FirstPostions, Integers, PrePendTo[]
• GetEnvironment # 938
• Integers, PrependTo and ContainsOnly
• Import support for WL packages
• IterationLimit
• LoadModule
• MantissaExponent[], FractionalPart[], CubeRoot[]
• PolynomialQ[], MinimalPolynomial[]
• Quit[], Exit[] #523, #814,
• RealDigits #891, #691, Interrupt, Unique
• RemoveDiacritics[], Transliterate[] #617
• Root #806
• Sign[], Exponent, Divisors, QuotientRemainder, FactorTermsList
• Speedups by avoiding inner classes, #616
• StringRiffle[], StringFreeQ[], StringContainsQ[], StringInsert
• SubsetQ and Delete[] #688, #784,
• Subsets #685
• SystemTimeZone and correct TimeZone #924
• System\`Byteordering and System\`Environemnt #859
• $UseSansSerif #908
• randchoice option for NoNumPyRandomEnv #820
• support for MATHICS_MAX_RECURSION_DEPTH
• option --full-form (-F) on mathics to parsed FullForm of input expressions

Ehancements and Bug fixes:

• speed up leading-blank patterns #625, #933
• support for iteration over Sequence objects in Table, Sum, and Product
• fixes for option handling
• fixes for Manipulate[x,{x,{a,b}}]
• fixes rule -> rule case for Nearest
• fixes and enhancements to WordCloud
• added StringTrim[]
• fixes URLFetch options
• fixes XMLGetString and parse error
• fixes LanguageIdentify
• fixes 2 <= base <= 36 in number parsing
• improved error messages
• fixes Check, Interrupt, and Unique #696
• fixes Eigenvalues, Eigenvectors #804
• fixes Solve #806
• proper sympolic expantion for Re and Im
• fixes a bug in the evaluation of SympyPrime #827
• clean up ColorData
• fixes unicode characters in TeX document
• update Django gallery examples
• fixes Sum and Product #869, #873
• warn when using options not supported by a Builtin #898, #645

Mathematica tracking changes:

• renamed FetchURL to URLFetch (according to the WL standard)
• renamed SymbolLookup to Lookup

Performance improvements:

• Speed up pattern matching for large lists
• Quadraditc speed improvement in pattern matching. #619 and see the graph comparisons there
• In-memory sessions #623

Other Changes:

• bump RecursionLimit
• blacken (format) a number of Python files and remove blanks at the end of lines
• Adding several CI tests
• Remove various deprecation warnings
• Change shbang from python to python3
• Update docs

Backward incompatablities:

• Support for Python 3.5 and earlier, and in particular Python 2.7, has dropped
• The graphs module (for Graphs) has been pulled until Mathics supports pymathics and graphics using networkx better. It will reappear as a pymathics module.
• The natlang (for Natural Language processing) has also been pulled. The problem here too is that the pymathics mechanism needs a small amount of work to make it scalable, and in 1.0 these were hard coded. Also, both this module and graphs pulled in some potentially hard-to-satisfy non-Python dependencies such as matplotlib, or NLP libraries, and word lists. All of this made installation of mathics harder, and the import of these libraries, natlang in particular took some time. All of this points to having these live in their own repositories and get imported on laziliy on demand