I started using tracking.js https://trackingjs.com/ to track colors. It works well out of the box, though documentation could be better. *documentation snob* I combined it with paper.js to create sinusoidally hovering agents. Watch the mooviee!
We want to use it in conjunction with Nicky's sim for example to use it to mark where agents should show up in the grid.