Getting a Shovel in Forager is actually pretty simple, but you will first need to acquire a skill called "Farming" in the lower-left part of the Skill Tree. This skill also unlocks the Mill and other farming options, so it's a nice skill to pick up anyway.
Once you have the Farming skill, head to a Forge and spend 4 x Iron Ingot and 4 x Wood to create a basic Shovel. This is a permanent tool that you will keep forever, but you can spend more resources if you want to upgrade it.
Upgrading your shovel increases the radius in which it allows you to dig with a single dig, as well as giving you a chance to find more rare resources when you dig.
How to use the ShovelWhen you craft a Shovel, use the mouse wheel or number of the keyboard to select it as your active item. When you do that, click on any tile that does not have a resource on it to dig. This will dig here, and potentially give you an item.
There are also a few dig spots in Forager, and these have a very high chance to yield a rare item when you dig on it. See the photo below; the dig spot is just a few blocks in front of the character. So always dig these once you have unlocked your Shovel!