Cosmoline, that's what the yellow is. You can clean it off (you're going to have to scrub). Just get some engine degreaser, and get out the steel wool or whatever you feel like scrubbing with. Make sure you dilute the degreaser, and be careful.
Also consider a drimmel with the metal brush attachment. It did wonders on my valve cover.