The best way to run Django on Google Cloud. This project is now on GitLab: https://gitlab.com/potato-oss/djangae/djangae
It's finally here! Thank you to everyone who has contributed over the last 6 years!
"DEFERRED_ITERATION_SHARD_INDEX" key to os.environ for callbacks running from defer_iteration_with_finalize
ensure_instance_included to ensure_instance_consistent
atomic() (when used as a context manager) now returns an object representing the current transactiondjangae.db.transaction.current_transaction() to return the same thing from inside an atomic() decoratorTransaction.has_been_read(instance), Transaction.has_been_written and Transaction.refresh_if_unread(instance) which allows writing safe transactional code.Transaction.has_already_been_read(instance) and Transaction.refresh_if_unread(instance) which allows writing safe transactional code.deferred.defer_iteration_with_finalize
Transaction.protect_read which prevents a specific instance being read inside a transaction.djangae.utils.retry to catch the Datastore's InternalError, and to better select wait times between attempts. Also improved the logging and prevented losing the source of the final exception when retrying fails.djangae.contrib.backup to use the new export API (the existing API was deprecated in Feb 2018). This adds a dependency of google-auth and google-api-python-client,
and also requires some manual permissions to be configured for the app service account. Existing djangae settings will be respected. Read https://cloud.google.com/datastore/docs/schedule-export for details on the new permissions required, and https://cloud.google.com/datastore/docs/export-import-entities
for an overview including differences between the two APIs.remove_duplicates to ListField and RelatedListField which removes duplicated elements and retain order while saving.ImportError when running ./manage.py runserver and the SDK is not already on the Python import path.str instead of unicode
<binary> identifier.OR query was used with an offset and some entities matched more than one branch of the OR query.models.get_models, models.get_apps and Options.module_name) in djangae.forms.fields.py.UniqueActionAdmin.model_choices() on djangae.contrib.uniquetool appdjangae.contrib.backup are created in a new, time-stamped directory to make managing backups easier. Only DJANGAE_BACKUP_ENABLED is required, all other backup settings are optional and the default is to create backups in the default cloud storage bucket. See the backup docs for details.None as well as remove unused index properties from the entityListField and SetField can now still be pickled when a non-callable default is specified. This was preventing them being used
in migrations.process_task_queues is now RAISE_ERROR. Tasks will no longer fail silently when processed using this method in unit tests.FileField and ImageField which accept an optional url_field argument to allow you to specify the name of another field on the model on which the URL to the file or image is cached.sleuth library in djangae.contribCSRF_USE_SESSIONS flagdjangae.utils.retry:
_initial_wait kwarg._max_wait time.@retry_on_error decorator for applying it to function definitions._retries argument to _attempts which is better APIdjangae.deferred.defer to fix issues with google.appengine.ext.deferred.defer
print() statements with logging.debug() in all unittestsmanage.py test execution--help outputinstance.relatedlistfield.all()[0] would retrieve all items before indexing, now it only grabs the firstinstance.relatedlistfield.values_list(...) which would die with an error in 0.9.10 and earlierdjangae/fields/allkeys-5.2.0.zip file to MANIFEST.in
TypeError would throw when calculating the ComputedCollationField value if the source value was unicodevalue_from_datadict in forms.fields.ListWidget return None when the value provided is None as the existing comment describes. This prevents an exception when save() is called on a ListWidget whose value is None.managepy) so you can use use gcloud compatible app.yaml files (which cannot contain an app_id). Override with --app_idclearsessions view to tasks and admins onlysleep() time in djangae.utils.retry which was sleeping in ns rather than ms
RelatedSetField and RelatedListField.djangae.contrib.backups
Huge thanks to everyone who contributed to this release by writing code, documentation, submitting issues, reviewing code and providing support.
This release brings migration support, along with various performance fixes, querying fixes, a new contrib.processing module, and a new contains indexer. There are also improvements to the way task queues are processed in tests and various sandbox fixes.
IMPORTANT! THERE ARE BREAKING CHANGES IN THIS RELEASE
djangae.contrib.processing.mapreduce has been added to provide a Django-friendly API to mapreduce. The existing
djangae.contrib.mappers API has been reimplemented in terms of djangae.contrib.processing.mapreduce
DEFAULT_FILE_STORAGE setting to point to 'djangae.storage.BlobstoreStorage'.disable_cache behaviour.ComputedCollationField to generate correct ordering for unicode strings.DJANGAE_USE_LEGACY_CONTAINS_LOGIC = True in your settings!!
This will be removed in the next release of Djangae so you'll need to re-save your entities with this setting set to False before upgrading (see detailed release notes).failure_behaviour argument as appropriatefailure_behaviour to control what happens
if an exception occurs in a taskHuge thanks to everyone who contributed to this release by writing code, documentation, submitting issues, reviewing code and providing support.
This release brings a large number of bug fixes and preliminary support for Django 1.11!
Note, this includes a change to gauth which will require you update your imports and INSTALLED_APPS
The default TEST_RUNNER has also changed to djangae.test.DjangaeDiscoverRunner which sets up and tears down the App Engine testbed before/after each test. If you were doing this manually before you either no longer need to, or need to override TEST_RUNNER to restore Django's DiscoverRunner. If you are using Nose, then continue using the provided Nose plugin, nothing has changed here.
disable_cache decorator now wraps the returned function with functools.wrapsprefetch_related() now works on RelatedListField and RelatedSetField
user.is_authenticated instead of user.is_authenticated() when using Django >= 1.10.max_length and min_length validation support to ListField, SetField, RelatedListField and RelatedSetField.contrib.gauth.datastore and contrib.gauth.sql to contrib.gauth_datastore and contrib.gauth_sql respectively.
contrib.gauth.common.* to the parent contrib.gauth module. I.e. removed the .common part.
AppEngineSecurityMiddleware to work with Django >= 1.10_meta is used for pagination, when no custom order has been specified on the query set.@task_or_admin_only decorator to djangae.environment to allow restricting views to tasks (including crons) or admins of the application.DJANGAE_CACHE_ENABLED=False. This fix now allows disabling the cache to be a successful workaround for https://code.google.com/p/googleappengine/issues/detail?id=7876.DJANGAE_CACHE_ENABLED=False when saving in a transaction or deleting them.RelatedListField and RelatedSetField
djangae.contrib.mappers.defer_iteration, so that it no longers gets stuck deferring tasks or hitting memory limit errors when uses on large querysets._deferred attribute in Django models for versions >= 1.10djangae.contrib.mappers.defer.defer_iteration to allow inequality filters in querysetsdjangae.contrib.mappers.defer.defer_iteration where _shard would potentially ignore the first element of the queryset__init__ methodRelatedSetFields and RelatedListFields weren't being converted to the same type as the primary key of the modelHuge thanks to everyone who contributed to this release by writing code, documentation, submitting issues, reviewing code and providing support.
This release of Djangae brings some small improvements and bug fixes. There is no breaking changes.
Huge thanks to everyone who contributed to this release by writing code, documentation, submitting issues, reviewing code and providing support.
SimulatedContentTypeManager. Now also allows patching ContentType manager in migrations.--app_id flagInvalidSenderError when trying to send an email, Djangae now logs the 'from' address which is invalid (App Engine doesn't include it in the error).UnicodeDecodeError if a query contained a non-ascii character.TaskQueue using the test stub (including when using djangae.test.TestCase.process_task_queues) would flush all task queues../manage.py runserver not working with Django 1.10 and removed a RemovedInDjango110Warning message at startup.--nothreading functionality to runserver (this went away when we dropped support for the old dev_appserver)dumpurls command had stopped working due to subtle import changes.get_serving_url to get the correct url for serving images from Cloud Storage.get_serving_url which would add an entity group to any transaction in which it was called (due to the Datastore read done by get_serving_url).get_serving_url call inside CloudStorage url method.get_serving_url call inside BlobstoreStorage url method when file is bigger than 32MB.gauth middleware to update user email address if it gets changedThis release of Djangae brings some small improvements and bug fixes all around. The namespaces feature of Datastore DB backend is no longer considered experimental.
0.9.6 is also a last release before we bring support for Django 1.10 (coming later this week!) and some breaking changes to ListField and SetField filtering.
Huge thanks to everyone who contributed to this release by writing code, documentation, submitting issues, reviewing code and providing support.
__init__.--install_deps flag to the runtests.sh script to allow triggering of dependency installation without having to delete the SDK folder.--install_sdk flag to both the runtests.sh script and to the install_deps.py script in the bundled 'testapp'.count() method on ShardedCounterField is deprecated because its function was ambiguous or misleading and was often mistakenly used instead of value(). It is replaced with a shard_count() method.djangaeidx.yaml file which can be distributed. The indexes in this file are combined in memory with the ones from the project root's djangaeidx.yaml. This means that a user of your app will not be required to run queries to generate indexes or manually add them to their project file.djangaeindx.yaml files when on production.DJANGAE_CREATE_UNKNOWN_USER is False.DJANGAE_CREATE_UNKNOWN_USER to True to match the original behaviour.test_ordering_on_sparse_field).upload_to argument to FileField would result in a broken "./" folder in Cloud Storage.runserver command to work.install_deps.py script in the bundled 'testapp' where it would always re-install the App Engine SDK, even if it already existed.This release of Djangae brings support for Django 1.10 and breaking changes to queryingListField and SetField filtering.
To update to 0.9.7 version of Djangae, you need to update your code around querying the ListField and SetField fields. __isnull lookup has been replaced with __isempty, __exact with __contains and __in with __overlap. Detailed instructions about how querying of these fields work now can be found here: http://djangae.readthedocs.io/en/latest/fields/#querying-listfield-and-setfield
Huge thanks to everyone who contributed to this release by writing code, documentation, submitting issues, reviewing code and providing support.
ListField and SetField, which now works similiarly to PostgreSQL ArrayField. __isnull lookup has been replaced with __isempty, __exact with __contains and __in with __overlap. This is a breaking change, so stick to Djangae 0.9.6 or update your code.my_queryset.filter(pk__in=other_queryset) will use other_queryset.values_list('pk') rather than fetching the full objects.ListField and SetField.This release brings lots of bug fixes as well as features and improvements. Full details below.
The only breaking change will require you to add djangae.contrib.contenttypes to your INSTALLED_APPS after django.contrib.contenttypes -- we moved things around to clean up and prepare for Django 1.10 support, which is already under development!
Huge thanks to everyone who contributed to this release by writing code, documentation, submitting issues, reviewing code and providing support.
assert_login_required and assert_login_admin methods to djangae.test.TestCase.sys.path so that libraries in the application folder take precedence over libraries that are bundled with the SDK (with some hard-to-avoid exceptions).djangae.contrib.locking, for preventing simultaneous executing of functions or blocks of code.djangae.utils to djangae.environment.is_in_task(), task_name(), task_queue_name(), task_retry_count().ContentType patching to djangae.contrib.contenttypes. DJANGAE_SIMULATE_CONTENTTYPES setting has been removed, add djangae.contrib.contenttypes to INSTALLED_APPS instead. djangae.contrib.contenttypes needs to be after django.contrib.contenttypes in the INSTALLED_APPS order.JSONField behaviour in forms: it's properly validating JSON string before saving
it and returns json object, not string when accessed through cleaned_data.ListFormField.clean to return [] instead of None for empty values.None values.blob-key in BlobstoreFileUploadHandler easier by using content_type_extra. This removes
ugly hacks from before Django 1.7, and fixes issue with regex in BlobstoreFileUploadHandler not recognizing
filenames properly.RelatedIterator that fails when related iterated fields model is set as string.MapReduceTaskuses the db returned by the application router(s) unless explicitly passed.__iexact indexer where values containing underscores would not be correctly indexed. (Existing objects will need to be re-saved to be correctly indexed.)dumpurls command in documentationThis is our first release bringing support for Django 1.9, and dropping support for 1.7.
If you're still using Django 1.7 in your project:
Huge thanks to everyone who contributed to this release by writing code, documentation, submitting issues, reviewing code and providing support.
DeleteAsync and now tries to perform deletions in transactional batches of 25. This should result in improved performance but may introduce subtle changes in behaviour, please keep an eye out for issues. For more details take a look at the extensive comment in the DeleteCommand class for all the reasons why this is particularly tricky to get right and keep fast.InsertCommand (related to the unique constrains), performance improvements.DJANGAE_CREATE_UNKNOWN_USER has been added. This replaces the previous settings DJANGAE_FORCE_USER_PRE_CREATION and DJANGAE_ALLOW_USER_PRE_CREATION.
DJANGAE_CREATE_UNKNOWN_USER defaults to False, meaning you will have to create user objects in the database with matching email addresses to allow people to access your site. For existing projects, the auth backend will recognise the old auth-related settings.DJANGAE_CREATE_UNKNOWN_USER=True then a Django user object will be created automatically when a user accesses your site (if there is no matching user already).keep_parents=True in concrete model inheritance__month or __day. So things like datefield__month__gt=X work nowensure_instance_included with ensure_instance_consistent
ensure_instances_consistent for the multiple object case_target argument to defer_iteration in mappersRelatedListField or RelatedSetField (#607)JSONField fixes after removing SubfieldBase dependency - to_python added and default value not converted to string anymore