OpenStreetMap data to PostgreSQL converter
This release makes the new middle database format the default. If you have not switched already, you need to reimport your database to take advantage of that.
We have changed the way we are parsing the command line options. The new code uses the CLI11 library (a copy of which is included in the repository) and is much cleaner and also much stricter. You now get warnings (and sometimes errors) for many combinations of options that don't make sense. Please check the output from osm2pgsql and osm2pgsql-replication for such messages and fix your command lines accordingly. Note especially that duplicated options are not allowed any more. This can happen, for instance, when using osm2pgsql-replication which adds the database connection parameters (such as -d
) when it calls osm2pgsql.
If all goes well this will be the last release starting with a 1. We are planning for a version 2.0.0 in the second quarter of 2024. In that release we will remove all the functionality that has been deprecated. We will also remove support for the legacy database middle format and only support the new format introduced in version 1.9.0.
Further changes:
max_connections
settings any more.pg_stat_activity
table in the database.-x, --extra-attributes
did not work due to a wrong SQL command. This is fixed now.Many thanks to Thunderforest who supported development of the features in this release.
This is a relatively small but still important release.
The new middle table format has changed slightly: the tags
field can now be NULL
. This makes storage more efficient and indexing faster. The new middle format is now declared stable and production ready. To use it, use the command line option --middle-database-format=new
, in a future version of osm2pgsql this will become the new default. If you have used this option already with one of the 1.9.x versions of osm2pgsql you have to reload your database or use this SQL command to update the table: ALTER TABLE <name> ALTER COLUMN tags DROP NOT NULL;
, for <name>
use planet_osm_nodes
, planet_osm_ways
, and planet_osm_rels
or the equivalents if you are using a different table name prefix.
Other changes:
area
type and the add_row()
functions are deprecated if you use them. If you get this warning, read https://osm2pgsql.org/doc/tutorials/switching-from-add-row-to-insert/ .docs
directory is now called man
, because it only contains the man pages. All other docs are on the project web site.This release fixes a bug introduced in 1.9.0 with two-stage processing that will lead to crashes. If you are using any 1.9.x version, please upgrade to 1.9.2.
In one case we had some performance problems updating an osm2pgsql database with 1.9.1 due to the PostgreSQL query planning choosing a bad plan. This release contains a workaround for that problem.
We also improved the (experimental) generalizer code a bit:
run_sql()
command now can have either a single SQL statement in the sql
field (as before) or a list of SQL commands.run_sql()
command now has an optional transaction
field which can be set to true
to wrap the SQL commands in BEGIN/COMMIT.if_has_rows
fields on the run_sql()
command can be set to string with an SQL query. If that field is set, the SQL statement(s) in the sql
field is only run, if the SQL query returns at least one row.This release fixes some small issues with 1.9.0:
This release brings three new major features:
Other changes include:
spherical_area()
function for flex config files to calculate the area of a (multi)polygon on the sphere.--middle-with-nodes
option allows you to store all tagged nodes in the database (with their tags and location).Please note that this version drops support for implicit DB schema other than public. If you rely on implict user schemas or custom schema paths, you now must configure the schema to be used with the --schema
option.
To compile osm2pgsql some new libraries are needed, please see the README.md for details.
For more information on all new features and changes read the more extensive release notes for 1.9.0.
This release contains some fixes and minor changes.
osm2pgsql-replication
script so it works correctly with PostgreSQL schemas.pole_of_inaccessibility()
function.The largest change is the addition of much more flexible index support in the flex output. The table definitions have a new (optional) field called indexes
now which takes a list of index definitions. If the field is not there, we fall back to what we did before and create a GIST index on the only/first geometry column of a table. But you can also define any kind of index you want: define which index method (BTREE, GIST, ...) to use on which columns, define WHERE
clauses and expression indexes and much more. See the flex-config/indexes.lua
Lua config for some usage examples and the manual for all the details. You can also force osm2pgsql to always build the id indexes which are normally only built in slim mode.
The gazetteer output and the command line option --with-forward-dependencies
are deprecated in this release and will be removed soon. They were only needed for Nominatim which switched to using the flex output recently.
Here are the other changes:
pole_of_inaccessibility()
Lua function to generate reasonably good label points from polygons. (This function is currently marked as experimental, which means it can change without notice at any time.)As always there were lots of code cleanups across the board, but especially in code accessing the database and in the C++/Lua glue code to make it more flexible and easier to use internally.
This release has some small changes only:
osm2pgsql-replication
script now always expects the osm2pgsql
binary to be in same path as itself.--middle-schema=SCHEMA
to the osm2pgsql-replication
script which allows placing the replication status table in a schema other than PUBLIC (Thanks to @JakobMiksch).This release fixes a few small bugs in osm2pgsql and closes some gaps in the geometry processing code released in 1.7.0. It also contains some security-related fixes as a result of the security audit.
as_multipoint()
function to complement as_multilinestring()
and as_multipolygon()
.as_multipoint()
, as_multilinestring()
, and as_multipolygon()
will now always return single geometries if possible. Single geometries are always allowed where multi geometries are allowed, so this does't break anything.centroid()
function now works for all geometry types.length()
function to compute the length of a geometry in map units.reverse()
function to turn geometries around (can be useful for ways tagged with oneway=-1
).simplify()
function is now available for multilinestrings, too. (Not for polygons yet.)flex-config
directory has been updated for the new geometry handling capabilities.ST_IsValid()
on create_only
columns.For this version we rebuilt a lot of the code around geometry processing and around expire. The different parts -- creation of geometries from OSM data, transforming geometries (like merging and splitting linestrings) and finally writing them out in WKB format for import into the database -- are now well separated and tested on their own. And we added some functions for geometry processing, too. osm2pgsql can now calculate the centroid of a polygon and simplify linestrings using the Douglas-Peucker algorithm.
But the best part is that all of that new geometry goodness is now available from the Lua config files when using the flex output. There are many new ways of processing geometries from Lua:
add_row()
function which only allowed a limited set of operations. This function is still available for backwards compatibility, but there is a new function insert()
now which doesn't have this magic. Instead geometries are treated like any other data type giving you a lot more flexiblity. Check out the example config files addresses.lua
, generic.lua
, simple.lua
and geometries-using-insert.lua
in the flex-config
directory for some ideas on what can be done.In this version we enabled the bucket index for way nodes by default. This had been around for a while but you needed a command line option to enable it. After some positive feedback from the community we decided to make this the new default. It will be used on new imports (existing databases will keep using the old index). The new bucket index is much smaller and can save you hundreds of gigabytes of disk space. See https://osm2pgsql.org/doc/manual.html#bucket-index-for-slim-mode for the details.
And again a lot of code cleanups and some smaller bug fixes went into this release. To make writing tests easier we added a new BDD testing framework based on behave (https://behave.readthedocs.io/) and re-wrote a lot of the existing tests. Writing tests is now much easier and a lot less tedious.
There are also a bunch of changes to the osm2pgsql-replication script to make it easier to use.
This is the first version of osm2pgsql that needs a C++17 compiler. And there is a new dependency on the boost::geometry library.