Is your coolant low?
I believe this engine uses a wax pellet type IAC that opens and closes from the temperature of the flow of coolant. If coolant is low, the wax doesn't heat up and the idle stays high. If you have intermittent idle issues, it could be because the computer is sensing this and trying to compensate for it by lowering the RPM, and it can go back and forth.
Just an idea, based on a 3VZ-E at work that had a similar issue.