One of the biggest mistakes new puppy owners make is expecting their puppy to hold it for longer than he is physically capable. The general guideline for puppy “hold times” is that each month of age equates to an hour of “hold time,” so a two-month-old puppy can hold it for roughly two hours. There are exceptions to the rule; your puppy should be able to hold it for slightly longer period of time at night as he gets older and you pup will need potty breaks more frequently when he’s playing.
Remember that training is an ongoing process. You will never be completely finished. It is important to keep working on obedience training throughout the life of your dog. People who learn a language at a young age but stop speaking that language may forget much of it as they grow older. The same goes for your dog: use it or lose it. Running through even the most basic tricks and commands will help them stay fresh in your dog's mind. Plus, it's a great way to spend time with your dog.
It’s best not to punish the puppy for making a mistake. Instead, focus on praising appropriate bathroom behavior. If you catch your puppy in the act, interrupt it by saying “No!” in a firm voice or clapping your hands loudly, then take the puppy outside to finish going to the bathroom. Never swat your puppy, yell at it, or try to rub its nose in the mess.