Expanding on that, a lot of us have keys made to keep in the panniers as well as the left side pocket. Convenient, but not absolutely necessary unless you need to jump start the bike.
The PA has "unlockable" panniers. I keep my left side locked with all my recovery gear, my right side is empty and stays unlocked to put my helmet in at stops.
Ryan