Fascinated by streaming data pipelines, I have been looking at different ways to get data out of a relational database like Oracle and into Apache Kafka. I have presented about this topic at a number of conferences. There is a recording of the session from Oracle Code San Francisco during the week of OpenWorld 2017 together with Stewart Bryson.
The session is an easy introduction to kafka and event streaming for DBAs followed by more detailed explanations and demos of different methods to connect oracle to kafka:
- Kafka Connect JDBC
- Poor Man’s CDC with flashback query
- Dbvisit Replicate pureflow
- Oracle GoldenGate for Big Data
I have recently built a new session on top of this one where we explore streaming ETL with KSQL to perform aggregates and roll-ups on the fly and join streams from different sources together.