Polynote Versions Save

• New website and docs!
  • https://polynote.org has been revamped using mkdocs
  • A bunch of documentation has been added to the docs page
• Preliminary alpha support for Scala 2.13 (with Spark 3) (not yet ready for release) #1189
• Fix an issue which sometimes caused Polynote to reject all updates, "freezing" the notebook in an old state. #1201 #1203
• Safety and performance improvements to ReprsOf solving OOMs when handling huge collections. #1171
• Fix some concurrent editing bugs. #1086
• Fix a crash when insanely long lines are printed to console. #1187
• Support for output of stderr to the UI (only really for Python right now) #1215
• Improve Python performance (in some cases running in Polynote was ~10x slower than the REPL) #1194
• Implement isatty() to work around some issues where Python libraries check for this.. #1207
• Some Dockerfile improvements and documentation fixes (Thanks @holdenk !) #1179
• Fixed a docs typo! (Thanks @kuckjwi0928 !) #1200
• Various other bugfixes, dependency updates, etc.

• PySpark dependencies now configurable. #1175

  There are two new configuration parameters in the spark config, under the pyspark key:

  • distribute_dependencies (boolean), which toggles PySpark dependency distribution.
  • distribution_excludes (list), a list of packages to exclude from PySpark dependency distribution.

  Additionally, boto and h5py are now excluded from distribution by default.

• Fix bug where starting cell with "package" could erroneously trigger package-cell detection. #1174

Features:

• Can now cancel (and unqueue) tasks #1129
• Path is now prepopulated when creating notebook after clicking on a directory #1150
• HTML output from cells is now isolated in an iframe. #1140
  • Note that this does not make things more secure; it's merely useful to scope CSS rules so they don't affect the entire page.

Bugfixes:

• Fix for a double-typing bug causing odd behavior #1130
• Fix layout of the kernel pane when the symbol table is too large #1128
• Fix a couple issues handling renamed notebooks #1138
• Fix for LaTeX editor positioning #1149, including a performance fix for the text editor #1147
• Cell buttons now more obvious when pressed #1135
• Fix a bug preventing execution times from being displayed when a notebook was first loaded #1146
• Fix broken plotting when JS-invalid variable names are present in a cell's context #1139
• Fix broken viz cells handling non-numeric data #1148
• Cell execution timer now stops if the kernel dies #1143
• Set Python recursion limit in 3.7 to mitigate weird wontfix Python bug #1144
• Better state propagation for Python cells #764
• Focus editor after creating cell #1158
• Fix Open Kernels in About view #1155
• Fix issue preventing POLYNOTE_INHERIT_CLASSPATH environment variable from working in IntelliJ #1106
• Fix path entries for pkg_resources lib to account for new entries after activating the venv. #1169
• Fix Docker build issue (thanks @ghoto !) #1170

Wow, it's been a long time since our previous release! We've got a pretty big release lined up for everyone to enjoy 😄

Thanks to everyone who waited patiently for 0.4.0!

• Frontend Rewrite The frontend has been rewritten from the ground up. This change should be mostly transparent to users, but puts us on a much better footing for future improvements. The rewrite did resolve a few longstanding issues and add a few minor features:
  • Notebook tabs are now restored upon reload (stored in the browser). #451
  • Fix a bug with large notebook scrolling #923
  • Fixed the Kill kernel button in the Running Kernels UI #756
  • Symbol table now clears when the kernel is restarted #208
  • Whitespace in configuration is now trimmed #497
  • Cell status badges are shown after reload #449
  • When the editor is disconnected an obvious error is shown to the user #721 #701
  • Improvements to UI reconnection logic after server restarts #736
• New Plot Editor! #1025
  • Brand new plot editor which supports re-editing plots, pie plots, bubble plots, native histogram, coloring by a dimension in many plots, faceted/trellis plots!
  • Data tables (from the data browser) and data views (expando-thingy) can also be embedded persistently in the notebook!
  • No more inspection modal! It’s a cell in the notebook!
• You can now Move Cells #1068
  • Mouse over the cell and select the handle that appears on the left. Drag the cell up or down.
  • The state of the cell is also carried along with it.
• New Quick Inspect UI for Symbol Table #1075
  • Simply hover over a row in the Symbol Table to see a menu with a representation of the value, type information, etc.
• New Copy to Clipboard button added to cells with output.
• Support for Multiple Scala Versions at the same time! So far only 2.11 and 2.12, sorry. Blame Spark)
  • The Scala version can now be set per-notebook. It is no longer related to the server Scala version.
  • Distribution now ships with kernels for all supported Scala versions.
  • Which Scala version to use can also be configured with kernel.scalaVersion. It is automatically detected if not configured.
  • shapeless was also removed from our dependencies.
• Configuration for setting JVM options on Kernels. #695
• Improvements to dependency caching #1042
  • New UI treatment for choosing caching preferences for dependencies. There is a new ... button next to each dependency.
  • Behavior depend on the type of dependency:
    • Cache status of JVM dependencies specified via URL (e.g., s3 or http) can be individually toggled in the UI
    • Cache status of pip dependencies can be toggled in the UI, but doing so will just reset the virtualenv (i.e., it will affect all pip dependencies).
    • Cache status of coordinate dependencies are always cached.
• Matplotlib output now defaults to PNG format. It can be configured to SVG format as well using PolynoteBackend.output_format = 'svg' # or 'png'. #1037
• Polynote now creates the notebook directory if it doesn't already exist. #964 (thanks @akiyamaneko !)
• Stack traces are now saved into the notebook file #981
• Properly set PYTHONPATH on executors, fix regression causing pip dependencies not to be added to Spark automatically #1000
• Hyperlinks now clickable in text cells #106
• New SparkRepr for Array[Row] #999 (thanks @tmnd1991 !)
• Scalar seqs are now streamed as structs. #1058
• Added DEVELOPING.md (thanks @easel !)

Deprecations

• Configuration option behavior.kernel_isolation: never is no longer supported – spark kernels can no longer be launched in-process. never is now deprecated; it behaves the same way as spark (remote kernel iff notebook uses Spark) and will be removed in a future version.

Bugfixes

• Fix for some kernel launching issues (one causing duplicate kernels, another causing kernels to be interrupted erroneously) #996
• Python dependency virtualenv is now reset when a change is detected. #921
• Polynote now shows an error when Python dependencies are not found #450
• Fix bug preventing http and s3 dependencies from being cached #1027
• Fix bug running Polynote on Python versions below 3.7 #972 (thanks @akiyamaneko !)
• Added Java8 compile flag (thanks @Lanking !)
• Update Coursier to 2.0.0-RC5-6 which fixes an issue Coursier has resolving platform-native libraries #998, # 1004 (thanks @kamilkloch !)
• Py4j gateway is now created on the jep thread, allowing py4j to see dependency jars.
• Polynote now extracts notebook name from Zeppelin json when importing (thanks @Ewan-Keith !)
• Fix issues preventing deletion of a notebook #974
• Properly handle duplicated dependencies #1039
• Fix some bugs with the LaTeX editor #969

• Fix an issue with Python Data Classes
• Fix issues importing Zeppelin notebooks
• Fix some dependency resolution bugs (Thanks @JD557)
• Fix an issue improperly decoding URI components (Thanks @akiyamaneko)
• Other minor bug fixes

• Fix a regression which stopped certain error types from appearing in the UI
• Check Notebook read permission when loading a notebook.

• New experimental Dark Mode (feedback welcome!)
• Ability to directly paste images into text cells (image button on toolbar still doesn't do anything, but at least now it's possible to insert an image without it having to already have a URL)
• Notebooks are loaded incrementally (better responsiveness for large notebook files)
• Some tweaks/fixes for Scala completions and signature help
• Handle case when the browser generates continuation frames over the websocket

This replaces 0.3.8, which was found to have a few issues that weren't caught.

• Stability A bug that could cause data loss has been fixed in this release.
• Backups Polynote now writes a write-ahead-log (WAL) whenever it gets updates to a notebook. This WAL can be recovered using polynote.py recover /path/to/wal-file.wal. This is currently experimental.
• Backups Polynote now saves a copy of opened notebooks into the browser's local database. This can be used in case of catastrophic server-side data loss (but where the client has been working).
• New Added a "notebook runner", invoked with polynote.py run [OPTIONS] input-file input-files* that can run notebooks in headless mode
• Lots of fixes to the Plot UI (Thanks @JD557!)
• Handle naming collisions between Java and Python package imports. Python packages with JVM-like names no longer get swallowed by the Jep importer.

• Fixed bug: saving plots doesn't work because websocket connection dies for no apparent reason
• Fixed bug: Value of zero or false causes [object Object] in table view... sigh javascript
• Fixed issue: Plotting pandas DataFrame fails when column identifiers have non-string type (thanks @Baoqi!)
• Configurable port range for remote kernel comms (thanks @hadrienk!)
• Parallel downloads of JVM dependencies
• Allow complex data structure display in table view
• Fixed some edge cases in Scala code that defines classes/types/methods and uses them in later cells

• Fixed regressions:
  • Cancel button not working
  • Rename active notebook causes bad state / data loss
  • Configured listen interface was ignored
  • Crash in SQL interpreter
  • JavaScript error when attempting parameter hints
• Fix issue with launcher script
• Fix multiple series in bar chart (Thanks @JD557!)
• Surface error if kernel dies before finishing startup