Preheat your oven to 350 degrees.
Use a hand mixer, stand mixer or whisk to make the vanilla bake batter per package instructions.
Spoon the batter into a cupcake tin lined with paper cupcake liners (if you're using a silicone cupcake pan you do not need paper liners). Make sure that each cupcake well is filled ~2/3 full. This should make 16-18 cupcakes.
While the cupcakes are baking, make the jell-o. Follow the package instructions to make the jell-o but use 1/2 less cup of cold water than instructed for each flavor.
Bake the cupcakes until the tops just begin to brown and a toothpick comes out clean (about 18-22 minutes). Allow them to cool for 15-20 minutes before removing them from the pan to assemble the poke cake cupcakes.
To assemble the cupcakes, use a wooden skewer or straw to poke holes in the tops of the cupcakes. If using a wooden skewer, make about 6 pokes in cupcakes. If using a straw, make 4 pokes in the cupcakes (since the hole is larger).
Carefully spoon red jell-o into half of the holes and blue jell-o into the other half of the holes (you will have extra jell-o mixture that isn't used). Set the cupcakes in the fridge for 20-30 minutes to allow them to begin to set.
Top them with cool whip and sprinkles! Store in the fridge.