Data Engineering
Getting Data Out of Systems That Don't Have APIs
A core operational system with no API, no clean export path, and no integration options offered by the vendor.
Situation
A core operating system had no API, no clean export, and no integration options. Every vendor conversation came back to the same answer: the only path forward was a full system replacement, which would cost six figures and disrupt operations for months. The data the team needed was visible inside the application — they could see it on the screen — but it was effectively trapped. Reporting, automation, and any cross-system workflow that depended on this data was either being done by hand or not done at all.
Approach
We started with Power Automate as the extraction layer because it could navigate the application’s UI and capture what was on screen. As the volume grew, we migrated the extraction work to Python scripts running on a scheduled task scheduler, in some cases inside a virtual-machine environment so the work didn’t tie up a desktop. The Python approach was significantly faster, more reliable, and cheaper to maintain than UI-driven RPA — RPA-style automation has a lot of breakage points (a UI element moves, a button gets renamed, the script halts), whereas Python scripts that target the underlying data flow are dramatically more durable. The extracted data flowed into a downstream warehouse layer where it could be joined with the rest of the company’s data and used in reporting and automation.
Outcome
Full automation achieved without system replacement and without the migration cost. The data the team needed for reporting and downstream workflows became available reliably, and the system the vendors said couldn’t be integrated became, effectively, integrated.
Related work
Other engagements that touch similar problems.
Have a similar problem?
Most engagements start with a 30-minute conversation. Tell us what you are working on.