Postgresql upgrade checklist

Checklist for major postgresql database upgrade: shutdown standby servers on each standby perform pg_upgrade shutdown master perform pg_upgrade if pg_upgrade fails on postgres database with encoding mismatch, re-init database with initdb start master verify master is ok check if recovery.conf present in slave datadir start slaves one-by-one check if slaves are live and recovering if […]

Fixing pg_dump invalid memory alloc request size

I’ve encountered unusual problem while dumping one of the bigger PostgreSQL databases. Every try to run pg_dump resulted in this: sam@cerberus:~/backup$ pg_dump -v -c js1 >x pg_dump: Error message from server: ERROR: invalid memory alloc request size 18446744073709551613 pg_dump: The command was: COPY job.description (create_time, job_id, content, hash, last_check_time) TO stdout; pg_dump: *** aborted because […]

Oracle VARNUM/NUMBER encoding in C

Just a short example C implementation of Oracle VARNUM/NUMBER type encoding, in case you could not use library functions. I didn’t found it anywhere when needed (and afterwards I found it doesn’t helps me at all due to little ‘problem’ in ora :() and Oracle documentation is pretty unclear about how-to do it.   typedef […]

dmlquery initial release

Just released initial release of dmlquery tool, for doing DML queries on SQL database (or anything perl DBI abstraction layer can handle) via formated XML files. Very useful for XML pipelining. With this and XML/XSLT transformations you can do very powerful thinks, like whole ETL tools for extracting data from one source (database, web…), transform […]

Solving MAX(COUNT()) problem 2 – optimizations

In previous post I’ve tried to present my solution for solving max(count()) problem.  The solution was slightly suboptimal and I’ve needed to speed it up a bit, because I’m using it now in some statistical calculations and every millisecond is important. This is original solution. CREATE VIEW data_view_source AS SELECT DISTINCT ON (dp_id) dp_id, ds_id […]

SQL timeline and statistical computations

This is simple way how-to generate time table (list of consequencing timestamps, or timeline) in PostgreSQL. Nothing spectacular, but might help you ,when trying to do some time based statistical selects (can feel like OLAP :-)). Code: CREATE OR REPLACE FUNCTION gen_time_list( tim IN TIMESTAMP, tim_stop IN TIMESTAMP, step IN INT) RETURNS TABLE(ts1 TIMESTAMP, ts2 […]