Category Archives: database

Christmas Gift!

Merry Christmas!!

If you are using SQL Server or Oracle databases, I’m giving away my 1 hour of time for rest of this week for free of cost to  discuss how you can save $$$$ by migrating them to PostgreSQL database by keeping same features and achieve better performance.

you can shoot an email to denish.j.patel@gmail.com

Happy holidays!

Running Postgres in Docker

For last six months, I have been working on moving Postgres from bare metal & VM based  systems in  Docker. As of today, we have migrated a couple of mission critical Postgres DBs  (~ 2TB) on to Docker environment.

During the migration  journey, I have listed down some of the things to consider running Postgres production instances in to Docker environment.

  1.  Do not use default Postgres Docker image. Start with your own docker image from scratch.
  2.  Handle Postgres service shutdown gracefully in docker with SIGTERM
  3.  OS can be stored in Docker container
  4.  Data volume MUST be stored in persistent storage
  5. Use some kind of framework to manage docker containers
    • Apache Mesos & Aurora
    • OpenStack & Kubernetes
  6.  You can mount NetApp for backups and WAL files on to container
  7.  Make templates for resources for different kind of workloads
    • Aurora job for resource templates
    • postgresql.conf templates
  8.  Use static IPs for services ; DBs, Front end an backend servers
    • It will be easier to control access at container level for better security
    • failover is easy to manage with static IP for master DB server
  9.  Benchmark your existing system and compare with new system
    • Keep eyes on TPS using pgbench and benchmarksql
  10.  Monitoring solution for Postgres DB
    • collectd or other agent based monitoring
      • pg_stat_statements is very useful
    • Docker container should be monitored separately
      •  docker stats
  11.  Backup container for taking backups
  12.  Standby container for setting up standby jobs

I hope it will be useful for someone working on migrating Postgres into Docker environment!

10 commandments of Database Management

After having decade of experience managing small to large scale and/or varieties of database  systems, here is my first try to come up with the top ten commandments of database management !

  1. Thou shalt always learn database systems strengths and weaknesses
  2. Thou shalt choose appropriate database to store and process data to empower business
  3. Thou shalt always build systems to be resilient/cope with failures
  4. Thou shalt implement automated restore procedures to test backups
  5. Thou shalt always trend and monitor database performance and maintenance metrics
  6. Thou shalt  document and follow database change management procedure
  7. Thou shalt plan to upgrade database systems in timely manner
  8. Thou shalt always build tools to automate processes
  9. Thou shalt implement security policy and processes to secure data
  10. Thou shalt educate developers to write efficient code against databases

If your commandments list differs from mine, I’m interested to know your list. Please don’t hesitate to post in comment.  Thanks!