Database security suite. Database proxy with field-level encryption, search through encrypted data, SQL injections prevention, intrusion detection, honeypots. Supports client-side and proxy-side ("transparent") encryption. SQL, NoSQL.
Core:
AcraServer:
consistent_tokenization
is now used by default for tokenization #614
RETURNING
statement for INSERT/UPDATE/DELETE #610
RESET
packets for MySQL binary protocol #611
schema()
functions, thanks to the new contributor @jercheng #634
acra_decryptions_total
with labels: {"status": [ "success", "fail"], "type": [ "acrastruct", "acrablock", "acrablock_searchable", "acrastruct_searchable"]}
acra_encryptions_total
with labels: {"status": [ "success", "fail"], "type": [ "acrastruct", "acrablock", "acrablock_searchable", "acrastruct_searchable"]]}
acra_tokenizations_total
with labels: {"status": [ "success", "fail"], "token_type": "{token_type}"}
acra_detokenizations_total
with labels: {"status": [ "success", "fail"], "token_type": "{token_type}"}
acra_acrastruct_decryptions_total
acra_api_encryptions_total
AcraTranslator:
--http_api_enable
is now true
by default starting from 0.96.0 #627
AcraServer, AcraTranslator, AcraKeys:
tls_ocsp_*
/tls_crl_*
options if the options tls_ocsp_[client|database]_*
/tls_crl_[client|database]_*
were not specified #617
AcraRollback, AcraRotate:
AcraTokens:
AcraKeys:
acra-keys destroy
command supports destroying poison record symmetric keys and keypairs, searchable and storage keypairs and symmetric keys #625
acra-keys destroy
command supports destroying rotated key with the new flag --index=<index:int>
#641. You can find examples on the documentation page
acra-keys import
/acra-keys export
supports keystore v1 #629
acra-keys list
lists the rotated keys for keystore v1 & v2 with the new flag --rotated-keys
#636. The formats of listed keys are updated. You can find examples on the documentation page/Core:
In this release we deprecated Zones functionality and all flags and CLI parameters related to it. These flags will be removed in the next versions. Acra will warn about deprecations.
Acra Community Edition supports separate encryption keys linked to the ClientIDs
and allows to manage key switching via TLS certificates.
Acra Enterprise Edition supports more flexible mapping between users/apps and encryption keys via SQL variables.
--kms_credentials_path=<filepath>
- path to configuration file specific for KMS type--kms_type=[aws]
- type of KMS provider--keystore_encryption_type
- specifies type of keys encryption for keystore. Accepts env_master_key
, vault_master_key
, kms_encrypted_master_key
, kms_per_client
. Read description of types on documentation pages of appropriate tools, for example AcraKeymaker.--encryptor_config_storage_type=[filesystem|consul]
) to switch source and Consul specific flags. Read more on documentation page about encryptor config and acra-server's configuration description. #568
SELECT
queries and update WHERE
clauses to add support of filtering with consistent tokenization. #581
database_settings
. #532, #590
mysql
and postgresql
mysql
subsection has one option, case_sensitive_table_identifiers
, boolean, to configure whether table names should be considered case-sensitive when comparing with names in encryptor configcase_sensitive_table_identifiers
option mentioned above
Case-insensitive means the identifier is converted to lowercase before comparing with values from encryptor config, encryptor config should contain lowercase version of column/table name.
Case-sensitive means identifiers are compared with values from encryptor config "as is", encryptor config should contain exactly the same identifier as in database schema.--tls_db_sni
flag. Now only --tls_database_sni
is available. #564
--tls_ocsp_[database|client]_required
, --tls_[ocsp|crl]_[database|client]_check_only_leaf_certificate
, --tls_[ocsp|crl]_[database|client]_from_cert
, --tls_[ocsp|crl]_[database|client}_cache_size
, --tls_[ocsp|crl]_[database|client}_cache_time
, --tls_[ocsp|crl]_[database|client}_cache_size
. You can find all of these flags in documentation on pages related to appropriate tool, for example AcraServer. #564.--http_api_tls_transport_enable=[true|false]
new flag added to turn on accepting TLS connections instead of raw TCP. Works only together with --http_api_enable=true
. #550
Example projects and demos:
This release brings type awareness which improves transparent encryption on AcraServer. Type awareness means that it's possible to tell AcraServer what are the original data types for fields. During decryption, AcraServer will convert decrypted fields to their original data types. No need to change client application code to work with "binary data".
It's also possible to choose a default value for each data field if its decryption failed. AcraServer can send a a default value like "<encrypted data>
" instead of decryption errors, making developers' and users' life easier.
Core:
encryptor_config
with new parameters:
data_type
- specify data type expected by application. Accept str
, bytes
, int64
, int32
values. #515, #517
default_data_value
- specify a placeholder (default value) to replace data that couldn't be decrypted. #515, #517
response_on_fail
- specify action on decryption failure. Accepts ciphertext
(returns encrypted data as is), default_value
(returns values from default_data_value
parameter), error
(returns error as DB error with message like encoding error in column {column_name}
). #521, #533
tokenize
parameter in encryptor_config
and changed focus on token_type
parameter. Now is enough to specify token_type
parameter without tokenize: true
to turn on tokenization. Read more in the documentation. #527
acra-poisonrecordmaker
. It improves decryption due to omitting extra key generation and poison record recognition. #516
set
command. #534
IsForbidden
field from acra-censor’s logs. Read more here in notes. #508
list
command. #530
install_dev_deps
install required golang's dependencies for development and code generation. #531
Documentation:
Example projects and demos:
This release brings stability and performance fixes to AcraServer and AcraTranslator. It officially deprecates usage of AcraConnector in favour of TLS everywhere. Some default configuration params are changed in favour of more secure & better performance settings.
Core:
--poison_detect_enable
changed default value from true
to false
. #484
error
level for success cases (not detected poison record),
clarified context of messages. #487
--keystore_cache_on_start_enable
that turns on loading all keys into in-memory cache on startup. #497
--keystore_cache_size
parameter from -1
(which means no limits for cache) to 1000 (cache items). #497
acrastruct
to acrablock
in the encryptor_config. Now AcraServer
will use faster encryption by default. You can select which CryptoEnvelope to use in encryptor_config.
See AcraStructs vs AcraBlocks documentation, #485
null::<type>
type casts. #479
--tls_client_id_from_cert
changed from false
to true
. Now AcraServer require app's TLS certificates and map them to keys. #481
--securesession_id
, --acraconnector_tls_transport_enable
, --acraconnector_transport_encryption_disable
. #481
--securesession_id
, --acratranslator_tls_transport_enable
, --acraconnector_transport_encryption_disable
. #481
transport-connector
, transport-server
, transport-translator
. #481
Example projects and demos:
Core:
docker/acra-authmanager.dockerfile
, docker/acra-webconfig.dockerfile
files.acra-webconfig
and acra-authmanager
from docker/acra-build.dockerfile
file that is base image for all
other service's images.acra-webconfig
and acra-authmanager
from all docker/docker-compose.*.yml
files.POST
HTTP request method additionally to GET
for v2 API. Method GET
marked as deprecated and warns with log message.
Deprecated HTTP GET method was used. Please use HTTP POST method instead.
if was used #466.Infrastructure:
-tags netgo
flag, that forces usage of Go resolver to solve issues related to resolving hostnames.
between Docker containers. Updated acra-build.dockerfile
used as base image for all cossacklabs/acra-*
images (#452).--keystore=v1
for existing docker-compose files that caused errors (#452).Documentation:
Example projects and demos:
Binary
to LargeBinary
#463.Acra's documentation is now open-source and updated for this release. Please find use cases, usage scenarios, data flows, descriptions of security controls, cryptography deep dive, scaling and load balancing, optimisations and many more.
Check out the updated documentation.
Two components can provide searchable encryption functionality:
encryptor_config
for INSERT
and
UPDATE
queries, calculating hash and searching by hash for SELECT
queries, with per column configuration.Read more details in the Acra documentation section dedicated to Searchable encryption.
INSERT
and UPDATE
queries, and transparent demasking for SELECT
queries, with per column configuration.
Read more details in the Acra documentation section dedicated to Masking.Two components can provide tokenization functionality:
Read more details in the Acra documentation section dedicated to Tokenization.
AcraBlock is a symmetric cryptographic container and is faster and more compact than AcraStruct. It used on AcraServer side in transparent encryption, masking, tokenization, searchable encryption.
AcraTranslator supports AcraBlocks in encryption, searchable encryption and tokenization via gRPC and HTTP API.
Read more details in the Acra documentation section dedicated to AcraBlock.
Added new storage format for keys in KeyStore that cryptographically strong key integrity checks, additional tracking metadata simplifying key management, KMS integrations.
Read more details in the Acra documentation about difference between two versions.
All Acra services that work with encryption/intermediate keys can load master key ACRA_MASTER_KEY
from HashiCorp Vault.
Previously was supported only environment variables.
Read more details on our KMS integration page in the documentation.
Read more details on our Integrating AcraServer into infrastructure.
Added prepared statements support for MySQL. Now all transparent operations over the data works with prepared statements too.
Extended and refactored TLS related CLI parameters.
tls_client_id_from_cert
- switching to new mode with clientID extraction from certificates instead of handshakes with AcraConnector or static mode with --client_id
parameter.tls_ocsp_url
, tls_ocsp_client_url
, tls_ocsp_database_url
- URL of OCSP server to use, for acra-server
may be configured separately for both directions.tls_ocsp_required
- whether to allow "unknown" responses, whether to query all known OCSP servers (including those from certificate).tls_ocsp_from_cert
- how to treat URL listed in certificate (use or ignore, whether to prioritize over configured URL).tls_ocsp_check_only_leaf_certificate
- whether to stop validation after checking first certificate in chain (the one used for TLS handshake).tls_crl_url
, tls_crl_client_url
, tls_crl_database_url
- URL of CRL distribution point to use, for acra-server
may be configured separately for both directions.tls_crl_from_cert
- how to treat URL listed in certificate (use or ignore, whether to prioritize over configured URL).tls_crl_check_only_leaf_certificate
- whether to stop validation after checking first certificate in chain (the one used for TLS handshake).tls_crl_cache_size
- how many CRLs to cache in memory.tls_crl_cache_time
- how long cached CRL is considered valid and won't be re-fetched.Separated parameters for connections accepted from application/AcraConnector or established to database with TLS:
acra-server
's certificate: tls_client_cert
and tls_database_cert
(overrides tls_cert
).acra-server
's key: tls_client_key
and tls_database_key
(overrides tls_key
).tls_client_ca
and tls_database_ca
(overrides tls_ca
).tls_client_auth
and tls_database_auth
(overrides tls_auth
).Supports RETURNING
syntax in SQL queries with proper decryption data in the response.
--sql_parse_on_error_exit_enable
new flag that force acra-server
to stop query execution if can't parse SQL query.
By default, it is false
.
Improved encryptor config validation.
Deprecated --acrastruct_wholecell_enable
and --acrastruct_injectedcell_enable
flags and will be ignored.
Now acra-server
works as in InjectedCell mode.
Deprecated --tls_db_sni
parameter and replaced with tls_database_sni
.
Read more details on our Integrating AcraTranslator into infrastructure.
--acratranslator_client_id_from_connection_enable
flag turns on mapping TLS certificates to encryption keys with .audit_log_enable
- new parameter turns on cryptographically signed audit logging. Read more in the Acra documentation.acra-server
and acra-translator
will map client's certificates to proper encryption keys in KeyStore.tls_identifier_extractor_type
- new parameter that configures strategy of extraction metadata from certificates for mapping to clientID (default: distinguished_name
, another option: serial_number
).--log_to_console
- parameter turns on\off logging to stderr.--log_to_file
- parameter specify path to file for logs. May be used together with logging to stderr.New flags to generate new kind of keys for new features:
--generate_hmac_key
- flag turns on generation symmetric key for HMAC used in searchable encryption.--generate_log_key
- flag turns on generation symmetric key for cryptographically signed audit logging.--generate_symmetric_storage_key
- flag turns on generation symmetric key for encryption with AcraBlocks.--keystore
- specify version of KeyStore. Now supported v1
(default) and v2
(new) versions.New flags to generate encryption keys for TLS certificates:
--tls_cert
- specify client's TLS certificate to generate encryption keys. Should be used instead --client_id
flag.--tls_identifier_extractor_type
- switch type of ClientID extraction from TLS certificate. Supports distinguished_name
(default) and serial_number
values.--fs_keystore_enable
now is deprecated and ignored.acra-tokens
is a new command-line utility used for managing generated tokens with turned on tokenization. Tokens may be stored in BoltDB or Redis for now.
Read more details in the Acra documentation.
acra-backup
is a command-line utility used for storing and managing the keystore backups. Also, it helps to migrate keys from one KeyStore to another one by export
+ import
operations.
Read more details in the Acra documentation on acra-backup page.
acra-keys
is a command-line utility used for different keys operations especially for v2 keystore. It consists of several subcommands each of which is responsible for a separate functionality.
make help
)pkg
target with automatic detection of OS (use it instead of rpm
and deb
)dist
, temp_copy
docker_push
target replaced with docker-push
Core:
Breaking changes:
Introducing a new more flexible configuration format for AcraCensor rules. AcraCensor doesn't support the old format, all users should migrate (don't worry, it's a simple procedure).
Search through encrypted data
You now can run SQL queries over encrypted AcraStructs allowing users to search through sensitive data without exposing it. This feature is only available in Acra Enterprise version.
Transparent proxy mode
TLDR: Transparent proxy mode allows you to configure AcraServer to encrypt records in specific database columns without altering the application code.
The application flow doesn't need to change: application sends SQL requests through AcraConnector and AcraServer to the database. AcraServer parses each request, encrypts the desired values into AcraStructs, and passes the modified requests to the database. To retrieve the decrypted data, your application talks to AcraServer again: upon receiving the database response, AcraServer tries to detect AcraStructs, decrypts them, and returns the decrypted data to the application.
Transparent proxy mode is useful for large distributed applications where updating the source code of each client app separately would be complicated.
To enable this mode, you need to create a separate encryptor configuration file (acra-encryptor.yaml
) that describes which columns to encrypt and provide a path to it in the AcraServer configuration file (or via CLI params --encryptor_config_file=acra-encryptor.yaml
).
Read more details in the Readme and in the Acra documentation section dedicated to Transparent encryption.
AcraCensor – SQL firewall to prevent SQL injections
TLDR: Improved stability of AcraCensor, switched to more flexible rules' configuration.
Breaking changes: Introducing a new format for configuration files, the previous format is no longer supported, you should migrate to the new one.
New configuration file format allows configuring the allowlist and the denylist separately or simultaneously.
The allow
handler allows something specific and restricts/forbids everything else. The allowall
handler should be a final statement as that means that all the other queries will be allowed.
The deny
handler allows everything and forbids something specific. The denyall
means "block all queries!" (that haven't been allowed or ignored before).
For each handler, there are settings that regulate queries, tables, and patterns. The order of priority for the lists is defined by their position in the configuration file. The processing priority for each list is as follows: queries, followed by tables, followed by patterns.
Read more in AcraCensor docs.
Added version to the configuration file. This allows detecting an outdated configuration easily. From now on, AcraCensor supports explicit configuration version and logs errors if the configuration is not valid (#321).
Improved parsing of SQL queries with prepared statements (#303, #283).
Improved error handling for queries that AcraCensor can't parse (#291, #284).
Added ability to log unparsed queries to a separate log file for the debugging and configuration purposes. Sometimes AcraCensor can't parse all of the incoming queries and it is useful to have a separate log for them.
How to use it: Provide the path to the unparsed queries log file in the configuration file parse_errors_log: unparsed_queries.log
(#295).
Improved support of PostgreSQL queries ("RETURNING"
clause) and quoted identifiers (now you can use "tablename"
and WHERE "column"=1
) (#296).
Fixed the bug in QueryCapture log that caused duplicated of records in the log to appear (#318).
AcraServer
Fixed handling of null-size packets in PostgreSQL protocol (#286).
Fixed handling of setting a custom connection API port (#294).
Fixed handling of the plain text data response: if the database returns a plain text response, it is redirected "as is" (#305).
Fixed handling of casted placeholders in expressions like SELECT $1::type1::type2 FROM table1 WHERE column1=$2::type3::type4
(#328).
Improved code quality (some refactoring here and there) (#302, #301).
AcraServer, AcraTranslator, AcraConnector
Refactored logs and error messages got even more descriptive and user-friendly (#312, #299, #317).
Added on-start version logging to make it easier to understand which version is running (#319).
Added versioning for configuration files of each service (#322).
Updated some configuration parameters descriptions for better user-friendliness (please see our docs of AcraConnector and AcraServer for detailed descriptions of each parameter and usage examples) (#329).
AcraWriter
Updated AcraWriter for ActiveRecord (Ruby), fixed dependencies, added support of mysql2 adapter (#287).
Updated AcraWriter for Django (Python), fixed potential encoding issues (#293, #292).
Updated AcraWriter for C++, improved cpp codec usage (#290, #289).
Added bitcode for AcraWriter iOS and added Swift example project (#327, #326, #325, #324, #323, #323, #307).
Improved distribution of AcraWriter for Android, now it's available via Maven (#310).
Other
Infrastructure:
Example projects and demos:
iOS Swift example project that shows how to generate AcraStructs with and without Zones.
Android example project that shows how to integrate AcraWriter library into Android app using maven, and then to generate AcraStructs with and without Zones, and to decrypt them using AcraTranslator.
AcraCensor demo that shows how to configure AcraCensor for SQL injections prevention in OWASP Mutillidae 2 example app.
Protecting data in a Rails application demo based on AcraServer, PostgreSQL, and Ruby on Rails client application.
Protecting metrics in TimescaleDB demo based on AcraServer, TimescaleDB, and Grafana.
Transparent proxy mode demo that shows how to configure AcraServer in Transparent proxy mode to protect Django-based application.
Related blog posts:
Features coming soon:
Pseudonymisation: an early version of pseudonymisation library/plugin for Acra for transparent data pseudonymisation.
Cryptographically protected audit log: protection for logs against tampering.
Documentation:
Updated AcraServer documentation to describe Transparent mode in more details.
Updated AcraCensor documentation to describe the new configuration format and procedures for migration from the previous one.
Updated AcraWriter documentation for iOS and Android to reflect the improved installation ways.
Hotfix:
Fixed an issue in communication of AcraServer and PostgreSQL that causes AcraServer to stop processing connection due to an unexpected error in parsing packets. The issue occurred when last data in data row column from PostgreSQL comes with empty data (0 bytes).
Hotfix:
Fixed an issue in communication of AcraServer with some specific ORMs (xorm precisely) with MySQL database. In some cases, when database has plaintext data, AcraServer can't decrypt it (which is ok), but propagates decryption error and closes connection (which is wrong, it's fixed).
Core:
Key management
AcraServer documentation, AcraTranslator documentation.
AcraWriter
acrawriter.hpp
with dependency on Themis, placed in wrappers/cpp.
Read the usage guide and examples in examples/cpp folder (#270).Logging
Improved logs of AcraConnector and AcraServer: use Debug log level for all network errors (closed connection, unavailable network, etc) and use Error log level only for cases of certainly unexpected behavior (#275).
Improved startup logs: log process PID on start of AcraServer, AcraConnector, AcraTranslator, and AcraWebConfig (#275).
Fixed timestamps: do not overwrite logs' timestamps (#273).
Tracing with OpenCensus
traceID
that helps measure how much time it needs to perform a certain data processing functions (i.e. checking requests via AcraCensor, encrypting data, decrypting AcraStructs, etc.). Traces can be exported to Jaeger (#279, #276, #274).You can read more about tracing in our documentation in Tracing in Acra.
A blogpost about technical details, profits, and pitfalls during the implementation of traces is coming soon.
Other
Improved AcraServer's connection handling: stop accepting connections after error and stop AcraServer instead of trying to accept connections after the listening socket was closed (#275).
Improved AcraCensor's handling of prepared statements for PostgreSQL binary protocol (#280).
Improved handling of terminating packets (COM_QUIT for PostgreSQL and TerminatePacket for MySQL) to correctly handle the closing connections from clients (#275).
Refactored inner logic of AcraCensor: improved code quality and stability, added more tests that use more patterns (#268).
Infrastructure:
Ceased testing and supporting Go versions below 1.9. This will only affect the users who build Acra manually from sources. You can install the pre-built Acra components shipped for various Ubuntu, Debian, and CentOS distributives using Installation guide. Alternatively, you can check out our Docker images and Docker-compose files in docker folder (#277).
Tested Acra suite with PostgreSQL v11 and MariaDB v10.3 and updated docker compose examples and Acra Engineering Demo to use it (#278).
Published Acra load balancing demo: it illustrates some of the many possible variants of building high availability and balanced infrastructure based on Acra data protection suite components, PostgreSQL, and Python web application. In these examples we used HAProxy – one of the most popular high availability balancers today.
Updated AcraStruct Validator – an online tool that can decrypt your AcraStructs. AcraStruct Validator is useful for developers who build their own AcraWriters (to validate AcraStruct binary structure).
Features coming soon:
Running SQL queries over encrypted data: perform AcraServer-side lookups (search) over protected data.
Pseudonymisation: early version of pseudonymisation library/plugin for Acra for transparent data pseudonymisation.
Cryptographically protected audit log: protection for logs against tampering.
Documentation:
AcraWriter C++ has a short guide for installing and using AcraWriter for C++.
AcraRotate: added description and notes about "dry-run" mode.
Updated documentation for logging, collecting metrics, and tracing in Acra.
Many small fixes here and there to make your overall experience of using Acra's docs on a new platform distinctive and smooth ;)