Think of a Virtual Private Server as a go-between a virtual hosting/shared account and a dedicated server.
‘Virtual’ means that it is not real. Basically, it is still a shared hosting account but unlike a normal shared hosting account, each VPS has its own dedicated slice of the CPU, RAM, and disk space. So basically, what you do will not affect the other VPS on the machine. You can reboot your VPS anytime you want and you will have root access.
Think of it this way. A Dedicated Server is like owning and living in your own mansion, you can share or use it’s completely yours, it is up to you.
A Virtual Private Server is like living in an apartment or a hotel room. You have your own doors, walls, and private bathroom. You have the privacy of your own room and you more or less can do anything you want. Of course, you will share the infrastructure with the other residents in the apartment building.
A Virtual Hosting/Shared Account is like having a bed in a dormitory. You basically share a bathroom, a common area, and so on.
Of course, in a virtual hosting account, if you are the only customer on the machine, you can more or less use all the resources there is. If you share, as long as all the customers do not use more than what the server can handle, it does not matter.
A VPS, on the other hand, has strictly defined the slice of the CPU/RAM and Disk space, and most of the time, unless permitted, you cannot go beyond that allocation. The good news is that no matter how crowded the server is, you still have your minimum allocation. The bad news is that if your site requires more than what the VPS slice can handle, you will bear with it even if there are more resources that are unused on the server. Of course, some VPS application allows bursting to a reasonable limit but some don’t.