Showing posts with label development. Show all posts
Showing posts with label development. Show all posts

Tuesday, August 12, 2014

Web Application Database Design: Audit Trail . Tracking changes to column data value changes.

A departure from my typical gadget, general computing posts. Today, I'm going to discuss database auditing trail. Or, if you are searching from the web, "How to keep track of database column value changes."

So what is an Audit Trail (also commonly called an Audit Log)? It is basically a record keeping of changes in a database. For example, if a user updates a record, an audit log should record who made the changes and what the changes were. For some industries, this is an absolute requirement. There are many different ways to implement it and in this blog, I am going to share a technique I use for web applications.

Here is an example of why you need to have an audit trail or a history table of your database changes. I developed a B2B web app that sends materials to various retail stores. Often, the address changes and shipments are often lost. Usually, the client like to blame the developers for the missed shipments which could be very costly. However, with an an audit trail, the developer can proved that the wrong address was the result of bad input on behalf of the client.

A common method to audit changes in a database is to create a trigger that monitors INSERTS, UPDATE, DELETES. Here is an good example fro this website,simple-talk.

Unfortunately, most web applications usually only have a single user login for the app itself. Users are often stored in a user table. If your users are managed by a web front end with a user table, triggers won't work for this type of scenario.

Another common mistake is to make a duplicate history table which mimics the table you want to log. If you have multiple tables, you end making multiple history tables. To me, this is bad design.

My method is to make a singular history table of all the changes. Yes, there is overhead as there are often two transactions. One for the actual UPDATE/INSERT and one for the actual logging. However, I think the overhead is well worth it in most circumstances. This can be done as a trigger or a concurrent action for your application.

Here is an example schema of an audit trail log table in MySQL. It is a single table that can account for any database table you plan to use. It is an "Amorphous" table; meaning it is consolidated and links by reference to whatever you need to log. In my example,the Audit Log is a table called "AuditLog" for this discussion.



Let me explain some of the entity columns.
ParentID is the key the record you are logging.
ParentObject is the table you are referencing.
CreatedById is the user ID of the user and CreatedDate should be obvious to when the log was created.
FieldName refers to the table column.
DataType refers to the TYPE of the column: VARCHAR, DECIMAL, INT, etc.
OldValue refers to the original data and NewValue is the new data.
If your data is a larger, you can use OldText/NewText for large changes like TEXT.
Notes are internal notes you can flag such as deletes, inserts,etc.

Here is an example of a record that was updated. The id of that original record was 4420 and we know it is the Stores table that was updated based on the ParentObject.
The Field was the Address. As you can see, the old address was 15426 Emerald.. and it was updated by userid 52. You can simply do a JOIN to get more info on the user.




This works well for my needs. To restore a record, you can do an UPDATE with a JOIN.

UPDATE $ParentObject s SET s.`$FieldName` = A.OldValue JOIN AuditLog A ON s.Id = A.ParentId WHERE A.id = 179;



Here is another example. If you wanted to see the name changes of a particular user "124" from a "user" table, you can do a simple select like this: SELECT * FROM AuditLog WHERE ParentId = 124 AND ParentObject ='user'


idParentObjectFieldNameParentIdCreateDateCreatedByIdOldValueNewValue
1241username1242005年03月01日1J SmithJon Smith
1654username1242009年08月25日2Jon SmithJonah Smith
2547username1242010年01月10日1Jonah SmithJonathan Smith
3645username1242010年07月21日4Jonathan SmithJonathan Smith Sr.




So if you are using something like PHP and MySQL, you can log database changes and restore updates easily. This is a simple way to keep track of database changes.

Monday, May 28, 2012

Mongodb GUIs


In my day job, I tend to keep up to date with relevant technologies and one of the latest web buzzwords today is noSQL. CouchDB, Cassandra, mongoDB, and many other noSQL alternatives have been gaining popularity with the young folks. Like usual, it is my job to keep abreast. I've been keeping myself entertained with mostly mongodb because I think it is one of the easiest and quickest to learn.

If you are looking to explore mongoDB, there are some great GUI tools to get you started. Installing mongoDB is pretty trivial so I won't cover it here.




On OSX, one of the best program I've come across is MongoHub. It is a very pretty and intuitive application. Within a few minutes, I was able to get some tangible progress in evaluating mongoDB. It has the ability to import MYSQL table schema into a mongoDB collection. I imported a working mySQL db I have been working on and I was able to quickly make JSON queries just from glancing at the mongodb reference.

Instead of the normal “SELECT db_column FROM table WHERE db_col = value AND db_col2 =value2” , you use BSON like this:

db.COLLECTION.find({'key': 'value'})

Since we are not using the console, there is no need to invoke the mongodb find command, it is all gui driven. For my imported DB collection, I simply typed in my query expression like: {'City' : /^Con/i }

Instead of playing with some “Hello World” tutorials, I already had a working set of data for evaluation within 5 minutes of installation. There was no need for test records or test data, I had real working data that I was already comfortable with. Another 20 minutes, I was able to write some PHP scripts to query and display records.

MongoHub can be found here:
http://mongohub.todayclose.com/


I haven't found anything in Linux comparable to MongoHub but these two solutions worked for me: PHPMoAdmin and JMongoBrowser.

Once you have the PHP mongo driver installed, you can run a web-based admin script. It works and you can create collections and schemas rather quickly. It is PHP based and there was no configuration or mucking around.

PhpMoAdmin:
http://www.phpmoadmin.com/




The other GUI is JmongoBrowser. It is cross platform and Java based. Again, it works but there is nothing to write home about.

JmongoBrowser:
http://edgytech.com/jmongobrowser/



Both phpMoAdmin and JmongoBrowser installed in Ubuntu 12.04 without issues. They also run on MacOSX.

Subscribe to: Comments (Atom)

AltStyle によって変換されたページ (->オリジナル) /