Oct 24

Security & PgWest2010

Open Source databases still needs solid presence in enterprise world. One of the myths about open source databases is that they aren’t secure. To eliminate this myth we have to share our experience related to security aspect of PostgreSQL database. At OmniTi , we help businesses to achieve PCI compliance or provide payment processor:PayPI . PayPI runs on PostgreSQL database.

I feel that Security aspect of the PostgreSQL database isn’t marketed as it should be. Who doesn’t want secured applications and databases ? The answer is everyone cares about security but we can’t able to find many talks about security in the open source conferences. PgWest 2010 has wide variety of talks. Thanks to conference committee to include number of talks about security aspects of application and database. One of them is my session that will talk about achieving PCI compliance with PostgreSQL. If your system transmit, process or store credit card data then it should be PCI compliance system. Please don’t miss this opportunity to share your knowledge & experience during the talk. Even if your system doesn’t need PCI compliance, we can still discuss about security in general and share knowledge about features related to security.
If you are planning to attend PgWest 2010 at San Francisco from Nov 2, 2010 to Nov 4, 2010, looking forward to see you in-person.
Sep 01

Outlining Surge Conference Sessions

OmniTi’s Surge conference is around the corner (Sept 30th) and there are plenty of industry leading speakers and most advance technology sessions to choose from list. Most probably I will be attending second day of two days conference. I spent a lot of time to decide on most interesting sessions to attend from wide variety of sessions. In the past, for other conferences, I found easy to choose from two parallel sessions but it’s not the case at Surge. It’s one of the most difficult task to choose one from two parallel talks.
I hope my list will help you to outline your sessions or at least encourage you to think now!
This is my outline of sessions to attend at Surge Conference on Day -2:
  • Enterprise solutions from commodity components: The Promise and the Peril – Bryan Cantrill
  • Don’t bet the farm on your cache – Brian Akins
  • Top 10 Lessons Learned from Deploying Hadoop in a Private Cloud – Rod Cope
  • Design for Scale – Patterns, Anti-Patterns, Successes and Failures – Christopher Brown
  • From disaster to stability: scaling challenges of my.opera.com -Cosimo Streppone
  • Availability, the Cloud and Everything -Joe Williams
Apr 01

PostgreSQL Recovery Manager – PG_RMAN

During PgEast, I heard a lot about making PostgreSQL acceptable at more enterprise shops. I firmly believe that PostgreSQL has potential but lack of marketing perspective during the promotion of product and tools. Anyways, let’s start creating/testing/using and telling new tools those can be attracted by C-Level people!!
If you have Oracle background , you must know Oracle Recovery Manager – RMAN. Oracle has RMAN since Oracle7i (may be longer ). It has been enhanced during each release. It is widely used program at most of the Oracle shops. Obviously, Oracle puts lots of marketing efforts around RMAN features per release.
I came across a Google Code project called pg_rman, which is developed by NTT Open Source Software Center. It looks promising by looking at features list. If i compare these features with Oracle, it includes features comparable to Oracle9i RMAN feature list.

pg_rman features:
  • Ease of use. Backup and restore can be done with just one command.
  • Online full backup, incremental backup, and archive backup.
  • Backup compression. Unused area in pages are removed and only actual data are compressed with gzip.
  • Automatic backup maintenance. Backup and archive WAL files older than specified days are deleted automatically.
  • Backup validation. Backup can be validated with CRC checks
  • No transaction lost in restore. Configuration file generator for point-in-time recovery is supported.
Let's put hands together to test a new tool and help making PostgreSQL more advanced with adding these rich tools!!

I am going to test the pg_rman pretty soon!!
Jan 20

Secure your web from SQL Injections

Last couple of months, I came across two incidents of SQL injections due to most common errors by IT professionals.

  1. How To Hack A Brazilian Power Company
  2. SQL Injection discovered on Wall Street Journal
Both incidents are stunning and they were exploited by hackers with minimal effort. Little care during web application development could avoid both incidents.
If you are IT professional , you must know about SQL injections’ cause and remedy.
wikipedia provides detailed definition :

“SQL injection is a code injection technique that exploits a security vulnerability occurring in the database layer of an application. The vulnerability is present when user input is either incorrectly filtered for string literal escape characters embedded in SQL statements or user input is not strongly typed and thereby unexpectedly executed. It is an instance of a more general class of vulnerabilities that can occur whenever one programming or scripting language is embedded inside another. SQL injection attacks are also known as SQL insertion attacks.”

Most common SQL injections are:

1. Incorrectly filtered escape characters
  • Most common is Quote or Special character handling

2. Incorrect type handling

  • Brazil incident fall into this category

3. Vulnerabilities inside the database server

  • WSJ incident fall into this category : BAD password!

4. Blind SQL injection

  • Result of this attack is not visible to hacker but it can help to exploit more SQL injections.
  • conditional responses
  • conditional errors
  • execute long running queries

You can avoid these catastrophic SQL injections by little care during the web application coding such as using bind variables, verify input values and take care of escaping characters. Database level security such as password and access control also help to avoid SQL injections.

I hope this will save you from future SQL injection :-)

Oct 09

Postgres 8.4 – Replace regular index with Text Based Index

I came across a nice post by Peter Eisentraut that shows how text based index can be used to do direct lookup on the text fields.

Quick test proves that..


postgres=# d employees
Table "public.employees"
Column | Type | Modifiers
--------+---------+-----------
emp_id | integer | not null
name | text |
Indexes:
"employees_pkey" PRIMARY KEY, btree (emp_id)

postgres=# create index idx_employees_name on employees(name);
CREATE INDEX
postgres=# explain select * from employees where name like 'Denish%';
QUERY PLAN
-----------------------------------------------------------
Seq Scan on employees (cost=0.00..25.38 rows=6 width=36)
Filter: (name ~~ 'Denish%'::text)
(2 rows)

postgres=# explain select * from employees where name='Denish';
QUERY PLAN
---------------------------------------------------------------------------------
Bitmap Heap Scan on employees (cost=4.30..13.76 rows=6 width=36)
Recheck Cond: (name = 'Denish'::text)
-> Bitmap Index Scan on idx_employees_name (cost=0.00..4.30 rows=6 width=0)
Index Cond: (name = 'Denish'::text)
(4 rows)

postgres=# create index idx_employees_name_like on employees (name text_pattern_ops);
CREATE INDEX
postgres=# explain select * from employees where name like 'Denish%';
QUERY PLAN
--------------------------------------------------------------------------------------
Bitmap Heap Scan on employees (cost=4.31..13.78 rows=6 width=36)
Filter: (name ~~ 'Denish%'::text)
-> Bitmap Index Scan on idx_employees_name_like (cost=0.00..4.31 rows=6 width=0)
Index Cond: ((name ~>=~ 'Denish'::text) AND (name ~ Bitmap Index Scan on idx_employees_name_like (cost=0.00..4.30 rows=6 width=0)
Index Cond: (name = 'Denish'::text)
(4 rows)

Above example shows that we don’t need regular index on name field as text based index (idx_employees_name_like) fulfill the gap. However, if we do range query i.e select * from employees where name > ‘denish’ and name<'patel' then text based index will not be used. I don't think we ever run a range query on text based fields!.