Those froglets are looking cool!
The pinhead crickets available at most pet stores might work. They can be gutloaded and dusted with calcium powder or other nutrient powder appropriate for froglets. If the pinheads are too quick for the froglets, you can pinch off their hindmost legs to slow them down.
I did feed tiny silkworms to a couple of panda bear froglets (I got them from Mulberry Farms). They'd probably work for these gray tree frogs, too. I'm not sure how well the silkworms ship this time of year.
Some tiny mealworms now and then would probably be okay.
And then there's also netting tiny insects from tall grass or from a porch light at night. On the Fourth, we saw an adult gray tree frog feeding at our friend's porch light, which had attracted loads of insects--midges, mosquitoes, etc. A toad had come out from under the porch to feast, too.
I'm not sure if this would work with froglets, but adult gray tree frogs do seem to have a pretty aggressive feeding response to an insect's legs grasping their snout. I would hold a moth or butterfly or mealworm up to the tree frog and let the insect's legs grip the tree frog's mouth, and the tree frog would soon gobble the insect. Maybe this response helps the frogs feed in the dark, or feed on quick, flying things like moths? Perhaps the same technique could be used for the froglets using tweezers with very tiny tips to hold the small food items.
Good luck!