With all pets/guardians, make sure to have them out before BL/Hero. If you drop/cast them after BL/Hero is cast they won't benefit from it. All other temporary gains to AP, SP, Crit, Haste including potions are used before you drop/cast your pets/guardians.