A battery is a form of potential energy. To explain this, we’ll have to get a little technical.
The battery stores electrical potential energy when the electrons move from the cathode to the anode. It’s how the battery is charged.
When the electrons move in another direction, they convert this chemical potential energy to electricity in the circuit, thus discharging the battery. So, the battery is all potential energy.