Hi everyone!

Last week’s challenge asked about looking into the data, and specifically how to keep it in sync. Both for files and then for the application specific bits.

For the files, yes there are various cloud products that will take care of this provided you have a common account/tenant, and proper security controls in place. Yet, there is the on-prem scenario to look at as well, and you are more than likely not going to allow someone else into your network. Especially if you are acquiring a business unit and not the whole enchilada. There is also the old school way (ftp/rsync), but then the question is who is the source of truth?

For the other part, when I mentioned the application specific bits, I was describing apps that store data as binary blobs that are accessed from many places. Think of a database as the prime example. After all, you wouldn’t replicate the underlying database files used by an Oracle Database right?

While databases are a fairly common scenario, you will still have the initial sync to provide a common point of time reference, and then apply the changes as they are made. This can be done in one of three ways:

  • Reading/replaying a transaction log file - which will require getting the file over using the file syncing options mentioned above.
  • Capture the change as it is being made and replicate it to another system - Products like Oracle’s Golden Gate would be one example, and Debezium as another allow for hot replication.
  • Capture the data before it hits the application and make a second call to a target app - This would require a lot more complexity, but may be the only option if the app doesn’t have a transaction log or native replication support.

Each has their trade-offs when it comes to costs, security, complexity, feasability, latency, and sometimes even bandwidth.

Also, as one acquaintance has shared in the past, floppies and usb drives are also an option. :-D

cab