A public blockchain is stored on several independent nodes/computers (all participating nodes in the network). Each of these computers ensures a constantly updated and complete copy of the blockchain and is sometimes responsible for checking the correctness of the new transactions (data from the added blocks). 

Accordingly, the entire transaction history of a public blockchain can be viewed permanently, so that it can still be transparently traced at a later point in time when which transaction was made or received from which address.