ZyzomysPedunculatus' revenge (3.25.1)

This minor release is primarily intended to fix some minor issues that were introduced or detected following our 3.25.0 release, but it also includes some major improvements that we want to test and stabilize before our upcoming 4.0.0 release.

Features

• Our recent introduction of a clientside cache for document content now allows us to load and display a readable copy of a document before the most recent history has been fully loaded from the server. You might notice that your drive and some document typees are now displayed in a "DISCONNECTED" of "OFFLINE" state until they gets the latest history. For now this just means the loading screen is removed soon so you can start reading, but it's also an essential improvement that will become even more useful when we introduce the use of service-workers for offline usage.
• We've added an offline mode to the server so that anyone developing features in CryptPad can test its offline and caching features by disabling the websocket components of the server. Use npm run offline to launch in this mode.
• We spent some time improving the support ticket components of the administration panel. Tickets are now shown in four categories: tickets from premium users, tickets from non-paying users, answered tickets, and closed tickets.
• We also improved the readability of some of the server's activity logs by rounding off some numbers to display fewer decimal points. On a related note, log events indicating the completion of a file upload now display the size of the uploaded file.
• Errors that occur when loading teams now trigger some basic telemetry to the server to indicate the error code. This should help us determine the origin of some annoying teams issues that several users have reported.
• Users of the rich text editor should now find that their scroll position is maintained when they are at the bottom of the document and a remote users adds more text.

Bug fixes

• Shortly after deploying 3.25.0 we identified several cases in which its cache invalidation logic was not correctly detecting corrupted cache entries. This caused some documents to fail to load. We quickly disabled most caching until we got the chance to review. Since then, we've tested it much more thoroughly under situations which made it more likely to become corrupt. Our new cache invalidation logic seems to catch all the known cases, so we're re-enabling the use of the cache for encrypted files and most of our supported document types.
• We found that a race condition in the logout process prevented the document cache from being cleared correctly. We now wait until the asynchronous cache eviction process completes before redirecting users to the login page.
• We discovered that the postMessage API by which CryptPad's different iframes and workers communicate could not serialize certain error messages after recent changes. We've added some special logic to send such messages in a valid format as well as some extra error handling to better recover from and report failed transmissions.
• In cases where user avatars fail to load (due to network issues or 404s) the first letter of the user's display name will be displayed instead
• We found that shared folders were reconnecting to the server correctly after a network failure, however, some changes in the UI caused clients to incorrectly remain locked.
• Some recent refactoring of some styles caused some buttons on the login page to inherit bootstrap's styles instead of our custom ones.
• A third-party admin brought it to our attention that a library that was used for some development tests was being fetched via http instead of https, and was thus blocked by some of their local configuration parameters. We've updated its source to load via secure protocols only.
• The recent replacement of a link to our faq with a link to our documentation platform violated some security headers and prevented the link from loading. We've fixed the inline link with some code to open this link in a compatible way.
• Finally, we found a bug that caused custom colors in the slide app to revert to the default settings on page reloads. Custom slide colors should now be preserved.

To update from 3.25.0 to 3.25.1:

1. Stop your server
2. Get the latest code with git checkout 3.25.1
3. Install the latest dependencies with bower update and npm i
4. Restart your server

ZyzomysPedunculatus (3.25.0)

Zyzomys pedunculatus image courtesy of Wikimedia commons

Goals

This is the last major release of our 3.0.0 release cycle. We wanted to mark the occasion with some big improvements to keep everyone happy in case we need to take some more time to prepare our upcoming 4.0.0 release.

Update notes

This update introduces some major database optimizations that should decrease both CPU and disk usage over time as users request resources and prime an on-disk cache for the next time.

We've also introduce the ability to archive illegal or otherwise objectionable material from the admin panel assuming you possess the ability to load the content in question. It's also possible to restore archived content via an adjacent form field on the admin panel as long as it has not been permanently deleted. Due to a quirk in how ownership of uploaded files works, restored files will not retain their "owners" property. We hope to fix this in a future release.

We've also made some minor changes to the example NGINX config file provided in cryptpad/docs/example.nginx.confg, specifically in this commit. CryptPad will probably work if you don't apply these changes to your nginx conf, but some functional improvements depend on the exposed headers.

To upgrade from 3.24.0 to 3.25.0:

1. Update your NGINX config as mentioned above.
2. Stop your nodejs server.
3. Pull the latest code using git (from the 3.25.0 tag or the main branch)
4. Ensure you have the latest clientside and serverside dependencies with bower update and npm install.
5. Restart the nodejs server.

Features

• This release makes a lot of changes to how content is loaded over the network.
  • Most notably, CryptPad now employs a client-side cache based on the the indexedDB API. Browsers that support this functionality will opportunistically store messages in a local cache for the next time they need them. This should make a considerable difference in how quickly you're able to load a pad, particularly if you accessing the server over a low-bandwidth network.
  • Uploaded files (images, PDFs, etc.) are also cached in a similar way. Once you'd loaded an asset, your client will prefer to load its local copy instead of the server.
  • We've updated the code for our full drive backup functionality so that it uses the local cache to load files more quickly. In addition to this, backing up the contents of your drive will also populate the cache as though you had loaded your documents in the normal fashion. This cache will persist until it is invalidated (due to the authoritative document having been deleted or had its history trimmed) or until you have logged out.
  • We've added the ability to configure the maximum size for automatically downloaded files. Any encrypted files that are above this size will instead require manual interaction to begin downloading. Files that are larger than this limit which are already loaded in your cache will still be automatically displayed.
• We've also changed a lot of the UI related to encrypted file uploads and downloads:
  • Encrypted files can display buttons instead of the intended media under a variety of circumstances (if they are larger than your configured limit or if there is no applicable rendering mode). The styles for these buttons are now much more consistent with those found throughout the rest of the platform.
  • The same assets should now display progress bars when downloading and decrypting encrypted media.
  • When the same asset is embedded into a document in more than one location it used to be possible to trigger two (or more) concurrent decryption processes. We've modified the rendering process so that duplicates are detected and rendered simultaneously after the relevant assets have been decrypted (once).
  • We noticed that some old code to filter out forbidden content from rich text pads was interfering with encrypted media. We've clarified the filtering rules to preserve such content (audio, video, iframes) when it occurs within an acceptable context.
  • We've fixed some inconsistencies with media styles and functionality across different editors. Most types of media now allow you to right-click and choose to share (open that asset's share menu) or open it in a different context (in the file app or in the relevant editor where this behaviour is supported).
  • The file app has been greatly simplified. It now uses the same methods to render encrypted media as is used elsewhere, so it also displays progress and has a more consistent UI.
  • The file uploads/downloads table has also been improved somewhat:
    • Download progress is displayed for groups of items when downloading a folder from your drive.
    • We found and removed a hard-coded translation from the table's header.
• In keeping with the theme of network traffic and files we've also made some improvements to policies for users' storage:
  • Users should now be prompted to trim the history of very large documents when viewing them, saving space for the server operator as well as freeing up some of the user's quota.
  • Users will also be prompted to use similar functionality available through the settings page when the history of their drive and other account-related functionality is consuming a significant amount of their quota.
  • Documents that you own used to be automatically added to your drive when viewed if they weren't already present. This was originally intended as an integrity check and a means to recover from incorrectly removed entries in your drive, however, as we now support the removal of owned elements from your drive without destroying them this only serves as an annoyance. As such, we have dropped this functionality.
  • The whiteboard editor allows users to insert encrypted images into whiteboards, but only up to a certain size. Before it would just warn you that your image was too large. Now it provides the actual size limit that you've exceeded.
  • The prompt to store uploads in your drive is now suppressed when uploading images via the support ticket panel.

Bug fixes

• This release includes a fix for a very severe bug in Chrome and its derivatives where attempting to open a URL from within our sandboxing system would crash the browser entirely. This version works around the problem by not doing that.
• We've improved offline detection such that "offline" status is specific to particular resources like your drive, teams, and shared folders rather than treating your account as simply "online or offline".
• We've optimized one of our less style sheet mixins that was used in a lot of places at a more specific scope than was necessary. This resulted in more time compiling styles and higher storage space requirements for the css cache in localStorage.
• A small helper function that was intended to stop listening for enter and esc keypresses after closing a modal was overly zealous and stopped listening after any keypress. This made it so that any prompt with an input field did not correctly submit or cancel when pressing enter or esc after typing some text.
• Various browsers now require the request for the permission to send notifications to originate from a "click" event, so CryptPad now opens a dialog prompting you to allow (or disallow) permission if you haven't already made that decision.
• Modern browsers commonly prevent tabs from opening new windows unless you've explicitly enabled that behaviour (it's an important feature), however, in some cases the indication that a new tab was blocked can be very subtle and some of our users did not notice it. We now check whether attempts to open a new tab were successful, and prompt the user to enable this behaviour so that CryptPad can perform regular actions like opening a pad from the drive.
• After some deep investigation we identified a number of scenarios where contact requests would behave incorrectly, such as not triggering a notification. Contact requests should now be much more stable. On a related note, it's now possible to cancel a pending contact request from the concerned user's profile.

YunnanLakeNewt (3.24.0)

Image courtesy of Wikimedia commons

Goals

We are once again working to develop some significant new features. This release is fairly small but includes some significant changes to detect and handle a variety of errors.

Update notes

This release includes some minor corrections the recommended NGINX configuration supplied in cryptpad/docs/example.nginx.conf.

To update from 3.23.2 to 3.24.0:

1. Update your NGINX config to replicate the most recent changes and reload NGINX to apply them.
2. Stop the nodejs server.
3. Pull the latest code from the 3.24.0 tag or the main branch using git.
4. Ensure you have the latest clientside and serverside dependencies with bower update and npm install.
5. Restart the nodejs server.

Features

• A variety of CryptPad's pages now feature a much-improved loading screen which provides a more informative account of what is being loaded. It also implements some generic error handling to detect and report when something has failed in a catastrophic way. This is intended to both inform users that the page is in a broken state as well as to improve the quality of the debugging information they can provide to us so that we can fix the underlying cause.
• It is now possible to create spreadsheets from templates. Template functionality has existed for a long time in our other editors, however, OnlyOffice's architecture differs significantly and required the implementation of a wholly different system.
• One user reported some confusion regarding the use of the Kanban app's tag functionality. We've updated the UI to be a little more informative.
• The "table of contents" in rich text pads now includes "anchors" created via the editor's toolbar.

Bug fixes

• Recent changes to CryptPad's recommended CSP headers enabled Firefox to export spreadsheets to XLSX format, but they also triggered some regressions due to a number of incompatible APIs.
  • Our usage of the sessionStorage for the purpose of passing important information to editors opened in a new tab stopped working. This meant that when you created a document in a folder, the resulting new tab would not receive the argument describing where it should be stored, and would instead save it to the default location. We've addressed this by replacing our usage of sessionStorage with a new format for passing the same arguments via the hash in the new document's URL.
  • The window.print API also failed in a variety of cases. We've updated the relevant CSP headers to only be applied on the sheet editor (to support XSLX export) but allow printing elsewhere. We've also updated some print styles to provide more appealing results.
• The table of contents available in rich text pads failed to scroll when there were a sufficient number of heading to flow beyond the length of the page. Now a scrollbar appears when necessary.
• We discovered a number of cases where the presence of an allow list prevented some valid behaviour due to the server incorrectly concluding that users were not authenticated. We've improved the client's ability to detect these cases and re-authenticate when necessary.
• We also found that when the server was under very heavy load some database queries were timing out because they were slow (but not stopped). We've addressed this to only terminate such queries if they have been entirely inactive for several minutes.
• It was possible for "safe links" to include a mode ("edit" or "view") which did not match the rights of the user opening them. For example, if a user loaded a safe link with edit rights though they only had read-only access via their "viewer" role in a team. CryptPad will now recover from such cases and open the document with the closest set of access rights that they possess.
• We found that the server query "IS_NEW_PAD" could return an error but that clients would incorrectly interpret such a response as a false. This has been corrected.
• Finally, we've modified the "trash" UI for user and team drives such that when users attempt to empty their trash of owned shared folders they are prompted to remove the items or delete them from the server entirely, as they would be with other owned assets.

XerusDaamsi reloaded (3.23.2)

A number of instance administrators reported issues following our 3.23.1 release. We suspect the issues were caused by applying the recommended update steps out of order which would result in the incorrect HTTP header values getting cached for the most recent version of a file. Since the most recently updated headers modified some security settings, this caused a catastrophic error on clients receiving the incorrect headers which caused them to fail to load under certain circumstances.

Regardless of the reasons behind this, we want CryptPad to be resilient against misconfiguration. This minor release includes a number of measures to override the unruly caching mechanisms employed internally by two of our most stubborn dependencies (CKEditor and OnlyOffice). Deploying 3.23.2 should force these editors to load the most recent versions of these dependencies according to the same policies as the rest of CryptPad and instruct clients to ignore any incorrect server responses they might have cached over the last few updates.

This release also includes a number of bug fixes which had been tested in the meantime.

Other bug fixes

• We removed a hardcoded translation pertaining to the recently introduced "snapshot" functionality.
• Inspection of our server logs revealed a number of rare race conditions and type errors that have since been addressed. These included:
  • multiple invocations of a callback when iterating over the list of all encrypted blobs
  • a type error when recovering from the crash of one of the database worker processes
  • premature closure of filesystem read-streams due to a timeout when the server was under heavy load
• A thorough review of our teams functionality revealed the possibility of some similarly rare issues that have since been corrected:
  • it was possible to click the buttons on the "team invitation response dialog" multiple times before the first action completed. In some cases this could result in attempting to join a single team multiple times.
  • it was also possible to activate trigger several actions that would modify your access rights for a team when the team had not fully synchronized with the server. Some of the time this was recoverable, but it could occasionally result in your team membership getting stuck in a bad state.

We've implemented some measures to correct any team data that might have become corrupted due to the issues described above. Access rights from duplicated teams should be merged back into one set of cryptographic keys wherever possible. In cases where this isn't possible your role in the team will be automatically downgraded to the rank conferred by the keys you still have. For instance, somebody listed as an administrator who only has the keys required to view the team will downgrade themself to be a viewer. Subsequent promotions back to your previous team role should restore your possession of the required keys.

To update to 3.23.2 from 3.23.0 or 3.23.1:

Perform the same upgrade steps listed for 3.23.0 including the most recent configuration changes listed in `cryptpad/docs/example.nginx.conf...

1. Modify your server's NGINX config file (but don't apply its changes until step 6)
2. Stop CryptPad's nodejs server
3. Get the latest platform code with git
4. Install client-side dependencies with bower update
5. Install server-side dependencies with npm install
6. Reload NGINX with service nginx reload to apply its config changes
7. Restart the CryptPad API server

XerusDaamsi's revenge (3.23.1)

We discovered a number of minor bugs after deploying 3.23.0. This minor release addresses them.

Features

• On instances with a lot of data (like our own) the background process responsible for evicting inactive data could time out. We've increased its permitted duration to a sufficient timeframe.
  • This process also aggregates some statistics about your database while it runs. Upon its completion a report is now stored in memory until it is overwritten by the next eviction process. This report will most likely be displayed on the admin panel in a future release.
  • We now introduce some artificial delays into this process to prevent it from interfering with instances' normal behaviour.
• Instance administrators may have noticed that support tickets include some basic information about the user account which submitted them. We've been debugging some problems related to teams recently and have included a little bit of non-sensitive data to tickets to help us isolate these problems.
• We've added some additional text to a few places to clarify some ambiguous behavior:
  • When creating a shared folder we now indicate that the password field will be used to add a layer of protection to the folder.
  • The "destroy" button on the access modal now indicates that it will completely destroy the file or folder in question, rather than its access list or other parameters.

Bug fixes

• We received a number of support tickets related to users being unable to open rich text pads and sheets. We determined the issue to have been caused by our deployment of new HTTP headers to enable XLSX export on Firefox. These headers conflicted with the those on some cached files. The issue seemed to affect users randomly and did not occur when we tested the new features. We deployed some one-time cache-busting code to force clients to load the latest versions of these files (and their headers).
• We addressed a regression introduced in 3.23.0 which incorrectly disabled the support ticket panels for users and admins.
• We also fixed some layout issues on the admin panel's new User storage pane.
• Finally, we added a few guards against type errors in the drive which were most commonly triggered when viewing ranges of your drive's history which contained shared folders that had since been deleted.

To update from 3.23.0 to 3.23.1:

0. Read the 3.23.0 release notes carefully and apply all configuration changes if you haven't already done so.
1. Stop your server
2. Get the latest code with git checkout 3.20.1
3. Install the latest dependencies with bower update and npm i
4. Restart your server

XerusDaamsi (3.23.0)

Image courtesy of Wikimedia commons

Goals

We plan to produce an updated installation guide for CryptPad instance administrators to coincide with the release of our 4.0.0 release. As we get closer to the end of the alphabet we're working to simplify the process of configuring instances. This release features several new admin panel features intended to supersede the usage of the server configuration file and provide the ability to modify instance settings at runtime.

We also spent some time finalizing some major improvements to the history mode which is available in most of our document editors. More on that in the Features section.

Update notes

This release introduces some behaviour which may require manual configuration on the part of the administrator. Read the following sections carefully or proceed at your own risk!

Automatic database maintenance

When a user employs the destroy functionality to make a pad unavailable it isn't typically deleted. Instead it is made unavailable by moving it into the server's archive directory. Archived files are intended to be removed after another configurable amount of time (archiveRetentionTime in your config file). The deletion of old files from your archive is handled by evict-inactive.js, which can be found in cryptpad/scripts/. Up until now this script needed to be run manually (typically as a cron job) with node ./scripts/evict-inactive.js. Since this isn't widely known we decided to integrate it directly into the server by automatically running the script once per day.

The same eviction process is also responsible for scanning your server's database for inactive documents (defined as those which haven't been accessed in a number of days specified in your config under inactiveTime). Such inactive documents are archived unless they have been stored within a registered users drive. Starting with this release we have added the ability to specify the number of days before an account will be considered inactive (accountRetentionTime). This will take into account whether they added any new documents to their drive, or whether any of the existing documents were accessed or modified by other users.

If you prefer to run the eviction script manually you can disable its integration into the server by adding disableIntegratedEviction: true to your config file. An example is given in cryptpad/config/config.example.js. If you want this process to run manually you may set the same value to false, or comment it out if you prefer. Likewise, if you prefer to never remove accounts and their data due to account inactivity, you may also comment it out.

If you haven't been manually running the eviction scripts we recommend that you carefully review all of the values mentioned above to ensure that you will not be surprised by the sudden and unintended removal of any data. As a reminder, they are:

• inactiveTime (number of days before a file is considered inactive)
• archiveRetentionTime (number of days that an archived file will be retained before it is permanently deleted)
• accountRetentionTime (number of days of inactivity before an account is considered inactive and eligible for deletion)
• disableIntegratedEviction (true if you prefer to run the eviction process manually or not at all, false or nothing if you want the server to handle eviction)

NGINX Configuration update

After some testing on our part we've included an update to the example NGINX config file available in cryptpad/docs/example.nginx.conf which will enable a relatively new browser API which is required for XLSX export from our sheet editor. The relevant lines can be found beneath the comment # Enable SharedArrayBuffer in Firefox (for .xlsx export).

Quota management

Up until now the configuration file found in cryptpad/config/config.js has been the primary means of configuring a CryptPad instance. Unfortunately, as the server's behaviour becomes increasingly complex due to interest in a broad variety of use-cases this config file tends to grow. The kinds of questions that administrators ask via email, GitHub issues, and via our Matrix channel often suggest that admins haven't read through the comments in these files. Additionally, changes to the server's configuration can only be applied by restarting the server, which is increasingly disruptive as the service becomes more popular. To address these issues we've decided to start improving the instance admin panel such that it becomes the predominant means of modifying common server behaviours.

We've started by making it possible to update storage settings from the User storage section of the admin panel. Administrators can now update the default storage limit for users registered on the instance from the default quota of 50MB. It's also possible to allocate storage limits to particular users on the basis of their Public Signing Key, which can be found at the top of the Accounts section on the settings page.

Storage limits configured in this way will supercede those set via the server's config file, such that any modifications to a quota already set in the file will be ignored once you have modified or removed that user's quota via the admin panel. Admins are also able to view the parameters of all existing custom quotas loaded from either source.

How to update

Once you've reviewed these settings and you're ready to update from 3.22.0 to 3.23.0:

1. Modify your server's NGINX config file to include the new headers enabling XLSX export
2. Stop CryptPad's nodejs server
3. Get the latest platform code with git
4. Install client-side dependencies with bower update
5. Install server-side dependencies with npm install
6. Reload NGINX with service nginx reload to apply its config changes
7. Restart the CryptPad API server

Features

• As mentioned in the update notes, this release features a server update which will enable XLSX export from our sheet editor in Firefox. XLSX files are generated entirely on the client, so all information will remain confidential, it only required a server update to enable a feature in Firefox which is required to perform the conversion.
• We've also made some considerable improvements to the history mode available in most of our document editors. We now display a more detailed timeline of changes according to who was present in the session, and group contiguous modifications made by a single user. Our intent is to provide an overview of the document's history which exposes the details which are most relevant to humans, rather than only allowing users to step through each individual change.
• Another change which is related to our history mode improvements is support for "version links", which allow you to link to a specific historical version of a document while you scroll through the timeline of its modifications. You can also create named snapshots of documents which will subsequently be displayed as highlights in the document's timeline.
• Up until now we did not support history mode for spreadsheets because our sheet integration is sufficiently different from our other editors that our existing history system could not be reused. That's still the case, but we've invested some time into creating a parallel history system with a slightly different user interface tailored to the display of sheet history.
• Team owners and admins can now export team drives in the same manner as their own personal drives. The button to begin a full-drive export is available on the team's administration page.
• During the summer we experimented with the idea of providing preview rendering options for more of the languages available in the code editor. We were particularly interested in providing LaTeX rendering in addition to Markdown. Unfortunately, it turned out to be a more complex feature than we have time for at the moment. In the process, however, we made it easier to integrate other rendering modes in addition to markdown. For the moment we've only added a simple rendering mode for displaying mixed HTML, but we'll consider using this framework to offer more options in the future.
• While it might not be very noticeable depending on the size of the screen you use to view CryptPad we've spent some time making more of our interface responsive for mobile devices. You may notice this in particular on the modal menus used for sharing, setting access control parameters, and otherwise displaying alerts.
• We've also begun improving support for screen-readers by adding the required HTML attributes to input fields and related markup. We'll continue to make incremental improvements regarding this and other accessibility issues that were raised during the third-party accessibility audit performed several months ago. This audit was performed on behalf of NLnet foundation (one of our major sponsors) as a part of their NGI Zero Privacy-Enhancing Technologies fund.
• The share modal from which users can generate shareable links already detects whether you have added any contacts on the platform and suggests how you can connect with them if you have not. We added this functionality some time late in 2019 since the same modal allowed users share documents directly with contacts and this mode became the subject of many support tickets. As it turns out, many users are now discovering contact functionality via the access modal through which you can add users to a document's allow list or delegate ownership. Since this has become a similar point of confusion we've added the same hints to make it a natural entry-point into CryptPad's social functionality.

Bug fixes

• We noticed that it was not possible for document owners to remove the extraneous history of old documents when those documents were protected by an allow list. This was due to the usage of an incorrect method for loading the document's metadata, leading to a false negative when testing if the user in question had sufficient access rights.
• We also discovered an annoying bug in our filesystem storage APIs which caused the database adaptor to prevent scripts from terminating until several timeouts had finished running. These timeouts are now cancelled automatically so that the scripts stop running in a timely manner.

WoollyMammoth (3.22.0)

Image courtesy of Wikimedia Commons

Goals

We've been working on some long-term projects that we hope to deliver over the course of the next few releases. In the meantime, this release includes a number of minor improvements.

Update notes

To upgrade from 3.21.0 to 3.22.0:

1. Stop your server
2. Get the latest platform code with git
3. Install client-side dependencies with bower update
4. Restart the CryptPad API server

Features

• Contributors have helped by translating more of CryptPad into Finnish and traditional Chinese via our weblate instance
• We've updated the syntax highlighting code that we use throughout the platform to include Rustlang (and possibly other languages that have been updated in the meantime).
• You can now use ctrl-f in user or team drives to jump immediately to the search interface instead of possibly scrolling up to click on its entry in the sidebar.

Bug fixes

• Some of the special behaviour implemented for Org-mode in our code editor sometimes failed when the document was first changed into Org-mode.
• We now clear some minor personal preferences like whether certain tooltips had been dismissed when you log out.
• We identified and addressed a number of issues with teams that caused valid teams to not be displayed and team member rights to fail to upgrade until a full session reload.
• We now display the number of days before an unregistered user's documents are considered inactive in their drive instead of hardcoding "3 months".

VietnameseRhinoceros (3.21.0)

Goals

This release was developed over a longer period than usual due to holidays, our yearly company seminar, and generally working on some important software-adjacent projects. As such, we opted not to aim for any major features and instead introduce some minor improvements and address some users' complaints.

Update notes

We've had a few disgruntled administrators contact us about our apparent failure to provide a docker image or to otherwise support their preferred configuration. With that in mind, this is a periodic reminder that CryptPad is provided to the public under the terms of the AGPL (found within this repository in the LICENSE file) which implies on our part no warranty, liability, or responsibility to configure your server for you. We do our best to provide the necessary information to correctly launch your own instance of the software given our limited budget, however, all such files are provided AS IS and are only intended to function under the narrow circumstances of usage which we recommend within the comments of the provided example configuration files.

With that said, the vast majority of our community acts kindly and courteously towards us and each other. We really do appreciate it, and we'll continue to help you to the best of our ability. With that in mind, we're happy to announce that we've written and deployed a first version of our user guide, available at https://docs.cryptpad.fr. The work that went into this was funded by NLnet foundation as an NGI Zero PET (Privacy-Enhancing Technology) grant. We are currently working on two more guides intended for developers and administrators, and will deploy them to the same domain as they are completed. In the meantime we have begun to update our README, GitHub wiki, and other resources to reflect the current recommended practices and remove references to unsupported configurations.

If you're only reading this for instructions on how to update your instance from 3.20.1 to 3.21.0:

1. Stop your server
2. Get the latest platform code with git
3. Install client-side dependencies with bower update
4. Install server-side dependencies with npm install
5. Restart the CryptPad API server

Features

• We spent a little bit of time during our company seminar and implemented a first version of an automatically generated table of contents in our rich text editor. It is populated using header styles applied with the editor's dropdown menus, and can be hidden by clicking the "Outline" button in the app toolbar.
• We also made it possible to change the default behaviour of the Kanban tag filter via the settings page. You may choose to compound the selection of multiple tags as AND, resulting in the display of cards that have all the selected tags rather than the default OR behaviour which displays any card including any one of the selected tags.
• We've integrated a third-party Org-mode library into our code editor which features some fancy click-handlers that toggle the state of certain org-mode classifications.
• The search results interface which is present in individual and team drives has been improved such that it displays a spinner while a search is pending and that it indicates when there are no results for a given term.
• We've added a Japanese font (Komorebi-gothic) for use within the spreadsheet editor and have received and integrated Japanese translations from a contributor via our weblate instance (https://weblate.cryptpad.fr).
• Finally, we've modified some behaviour in individual and team drives, making it possible to move a shared folder to the trash where it was previously only possible to directly remove it from your drive.

Bug fixes

• We've corrected a minor server issue in which it would respond to requests to destroy non-existent files with an E_NO_OWNERS error, rather than an ENOENT (doesn't exist) error. The client code interpreted this as the file existing without them having the rights to delete it, rather than realizing that it no longer existed. This made it more difficult to remove files from your drive since destruction would fail rather than be interpreted as unnecessary.
• We now guard against race conditions in our internal write-queue library, preventing a rare occurrence of a type error triggered by unknown circumstances.
• We discovered that Firefox had enabled (by default) half of the functionality required to export sheets to an XLSX format. We interpreted the presence of this feature as sufficient cause to display XLSX as an export option, even though the export would fail if you tried to use it. The second half of the required functionality is available in Firefox, but requires specific HTTP headers to be sent by our server. We're currently testing the configuration parameters and expect to make XLSX export available on CryptPad.fr very soon, along with an update to our recommended configuration which would enable it on other instances.
• Lastly, we discovered an incompatibility betweeen our "safe links" behaviour and the process of redirecting users to log in or register to access specific functionality. Users that were redirected from pads accessed with safe links were redirected to that safe link whether or not they had imported the pad's keys into their newly created drive. This could result in a temporary loss of access to the pad, even though its credentials were still stored within their browser. We've corrected the redirect process to preserve the full document credentials for after you have logged in.

UplandMoa's revenge (3.20.1)

Once again we've decided to follow up our last major release with a minor "revenge" release that we wanted to make available as soon as possible. We expect to deploy and release version 3.21.0 on Tuesday, July 28th, 2020.

Features

• The markmap rendering mode which was recently added to markdown preview pane implements some click event handlers which overlap with our existing handlers which open the embedded mindmap in our full screen "lightbox". You can now use ctrl-click to trigger its built-in events (collapsing subtrees of the mindmap) without opening the lightbox.
• We've made a few improvement to user and team drives:
  • The list mode now features a "ghost icon" which you can use to create a new pad in the current folder, matching behaviour that already existed in grid mode.
  • We've also updated the search mode to display a spinner while your search is in progress. We also display some text when no results are found.
  • Team drives now open with the sidebar collapsed.
• Our rich text, code, slide, and poll apps now intercept pasted images and prompt the user to upload them, matching the existing experience of dragging an image into the same editable area.
• We've received new contributions to our Romanian translation via our weblate instance.

Bug fixes

• We identified some race conditions in our spreadsheet app that were responsible for some corrupted data during the period leading up to our 3.20.0 release, however, we wanted to take a little more time to test before releasing the fixes. As of this release we're moving to a third version of our internal data format. This requires a client-side migration for each older sheet which will be performed by the first registered user to open a sheet in edit mode, after which a page reload will be required. Unregistered users with edit rights will only be able to view older sheets until they have been migrated by a registered user.
• We now guard against empty mathjax and markmap code blocks in their respective markdown preview rendering extensions, as we discovered that empty inputs resulted in the display of "undefined" in the rendered element.
• We noticed and fixed two regressions in user and team drives:
  1. drive history had stopped working since the introduction of the "restricted mode" for shared folders which were made inaccessible due to the enforcement of their access lists.
  2. users with shared folders which had been deleted or had their passwords changed were prompted to delete the folder from their drive or enter its new password. The "submit" button was affected by a style regression which we've addressed.
• We've updated to a new version of lodash as a dependency of the linters that we use to validate our code. Unless you were actively using those linters while developing CryptPad this should have no effect for you.
• Finally, when users open a link to a "self-destructing pad" we now check to make sure that the deletion key they possess has not been revoked before displaying a warning indicating that the pad in question will be deleted once they open it.

To update from 3.20.0 to 3.20.1:

1. Stop your server
2. Get the latest code with git checkout 3.20.1
3. Install the latest dependencies with bower update and npm i
4. Restart your server

UplandMoa (3.20.0)

Upland moa image courtesy of Wikimedia commons

Goals

We've held off on deploying any major features while we work towards deploying some documentation we've been busy organizing. This release features a wide range of minor features intended to address a number of github issues and frequent causes of support tickets.

Update notes

This release features a modification to the recommended Content Security Policy headers as demonstrated in ./cryptpad/docs/example.nginx.conf. CryptPad will work without making this change, however, we highly recommend updating your instance's nginx.conf as it will mitigate a variety of potential security vulnerabilities.

Otherwise, we've introduced a new client-side dependency (Mathjax) and changed some server-side code that will require a server restart.

To update from 3.19.1 to 3.20.0:

1. Apply the recommended changes to your nginx.conf
2. Stop your server
3. Get the latest platform code with git
4. Install client-side dependencies with bower update
5. Reload nginx to apply the updated CSP headers
6. Restart the CryptPad API server

Features

• As noted above, this release features a change to the Content Security Policy headers which define the types of code that can be loaded in a given context. More specifically, we've addressed a number of CKEditor's quirks which required us to set a more lax security policy for the rich text editor. With these changes in place the only remaining exceptions to our general policy are applied for the sake of our OnlyOffice integration, though we hope to address its quirks soon as well.
• On the topic of the rich text editor, we also moved the print action from the CKEditor toolbar to the File menu to be more consistent with our other apps.
• The Kanban board that we use to organize our own team has become rather large and complex due to a wealth of long-term ideas and a large number of tags. We started to notice some performance issues as a result, and have begun looking into some optimizations to improve its scalability. As a start, we avoid applying changes whenever the Kanban's tab is not visible.
• We finally decided to file off one of the platform's rough edges which had been confusing curious users for some time. Every registered user is identified by a randomly-generated cryptographic key (the Public Signing Key found on your settings page). These identifiers are used to allocate additional storage space via our premium accounts, and we occasionally require them for other support issues like deleting accounts or debugging server issues. Unfortunately, because we occasionally receive emails asking for help with other administrators instances these keys were formatted along with the host domain in the form of a URL. As such, it was very tempting to open them in the browser even though there was no functionality corresponding to the URL. We've updated all the code that parses these keys and introduced a new format which is clearly not a URL, so hopefully we'll get fewer messages asking us why they don't work.
• We've made a number of small improvements to the common functionality in our code and slide editors:
  • We've merged and built upon a pull request which implemented two new extensions to our markdown renderer for Mathjax and Markmap. This introduces support for embedding formatted equations and markdown-based mind maps. Since these depend on new client-side code which would otherwise increase page loading time we've also implemented support for lazily loading extensions on demand, so you'll only load the extra code if the current document requires it.
  • The slide editor now throttles slide redraws so that updates are only applied after 400ms of inactivity rather than on every character update.
  • We've made a number of small style tweaks for blockquotes, tables, and embedded media in rendered markdown.
• Lastly, we've made a large number of improvements to user and team drives:
  • Search results now include shared folders with matching names and have been made sortable like the rest of the drive.
  • Inserting media in a document via the Insert menu now updates its access time, which causes it to show up in the Recent pads category of your drive.
  • Shared folders now support access lists. To apply an access list to a shared folder that you own you may right-click the shared folder in your drive, choose Access, then click the List tab of the resulting dialog. Enabling its access list will restrict access to its owners and any other contacts that you or other owners add to its list. Note, this access applies to the folder itself (who can view it or add to its directory), its access list will not be applied recursively to all the elements contained within which might be contained in other shared folders or other users drives.
  • In the interest of removing jargon from the platform we've started to change text from "Delete from the server" to "Destroy". We plan to make more changes like this on an ongoing basis as we notice them.
  • We've made a significant change to the way that owned files are treated in the user and team drives. Previously, files that you owned were implicitly deleted from the server whenever you removed them from your drive. This seemed sensible when we first introduced the concept of ownership, however, now that a variety of assets can have multiple owners it is clearly less appropriate. Rather than require users to first remove themselves as a co-owner before removing an asset from their drive in order to allow other owners to continue accessing it we now offer two distinct Remove and Destroy actions. Remove will simply take it out of your drive so that it will no longer count against your storage limit, while Destroy will cause it to stop existing for everyone. To clarify the two actions we've associated them with a trash bin and paper shredder icon, respectively.

Bug fixes

• Remote changes in the Kanban app removed pending text in new cards, effectively making it impossible (and very frustrating) to create new cards while anyone else was editing existing content or submitting their own new cards.
• Dropping an image directly into a spreadsheet no longer puts the UI into an unrecoverable state, though we still don't support image drop. To insert images, use the "Insert" menu. This was actually fixed in our 3.19.1 release, but it wasn't documented in the release notes.
• When a user attempted to open an automatically expiring document which had passed its expiration date they were shown a general message indicating that the document had been deleted even when they had sufficient information to know that it had been marked for expiration. We now display a message indicating the more likely cause of its deletion.
• We've spent some time working on the usability of comments in our rich text app:
  • When a user started adding a first comment to a document then canceled their action it was possible for the document to get stuck in an odd layout. This extra space allocated towards comments now correctly collapses as intended when there are no comments, pending or otherwise.
  • The comments UI is now completely disabled whenever the document is in read-only mode, whether due to disconnection or insufficient permissions.
  • The comment button in the app toolbar now toggles on and off to indicate the eligibility of the current selection as a new comment.
• We've fixed a number of issues with teams:
  • Users no longer send themselves a notification when they remove themself as an owner of a pad from within the Teams UI.
  • The worker process which is responsible for managing account rights now correctly upgrades and downgrades its internal state when its role within a team is changed by a remote user instead of requiring a complete worker reload.
  • The worker does not delete credentials to access a team when it finds that its id is not in the team's roster, since this could be triggered accidentally by some unrelated server bugs that responded incorrectly to a request for the team roster's history.
• We've fixed a number of issues in our code and slide editors:
  • The "Language" dropdown selectors in the "Theme" menu used to show "Language (Markdown)" when the page was first loaded, however, changing the setting to another language would drop the annotation and instead show only "Markdown". Now the annotation is preserved as intended.
  • A recent update to our stylesheets introduced a regression in the buttons of our "print options" dialog.
  • While polishing up the PRs which introduced the Mathjax and Markmap support we noticed that the client-side cache which is used to prevent unnecessary redraws of embedded media was causing only one instance of an element to be rendered when the same source was embedded in multiple sections of a document.
• The "File export" dialog featured a similar regression in the style of its buttons which has been addressed.
• We fixed a minor bug in our 3.19.0 release in which unregistered users (who do not have a "mailbox") tried to send a notification to themselves.
• We've added an additional check to the process for changing your account password in which we make sure that we are not overwriting another account with the same username and password.