Ha ha … man alive. Tonight I got my Transaction History code down from 4 hours to 4 minutes. 4 MINUTES!
How you ask? Well, it begins and ends with my stupidity. And the crazy thing is, even with code i wrote, the 4 hours was quicker than at the beginning. Strange.
A while back, when I was trying to speed up this Transaction History (Batching) functionality, i read up on stream() and flush(). This progressed to logout() and commit(). So I put them in, and as said, this sped up the operation (runs through a handful of files, which each have a few thousand records). But it wasn’t very quick. I mean, 4 hours is okay once, but I found myself needing it done quite a lot. Which might be a problem in and of itself.
Anyway, then, the other day, i discovered something.
Wait for it. It’s a kicker.
I hadn’t opened (.UseFile() is what i use lately) the files used in the batching before calling them in logout().
Like i said, crazy stupid.
What’s even more bizarre is that not opening the files, but calling logout() still had some effect on the speed. Anyway, chalk that one up to needing more cranial capacity. Such a simple thing, so obvious, yet i missed it for so long.