A book could probably be written on this- I'll try to cover the high points.
The WDT is a tool that can be used to help satisfy some system-level requirements, and those requirements are the primary motivation. They could be safety-related or arise from a desire to reduce some annoyance to the user. There are other tools such as clock monitors, external WDTs, redundancy, interrupts on access to improper memory locations, interlocks of various kinds, etc.
A WDT will not prevent anything- it can be used to help recover from a system upset. When a WDT timeout is triggered, you know something really bad has happened, but you don't know exactly what, so you must begin some kind of recovery operation. The processor could have gone off and done all sorts of things- corrupted memory (RAM or EEPROM), re-written SFRs to improper values, set outputs to undesirable states etc. The first thing you need to do is to put the system in a 'safe' state if that is a concern, and if it is possible. For example, a 10kW heater should be switched 'off' in most cases (there are cases where 'on' is better). Some things (consider a drone autopilot) may not have a safe passive state, and you may have to resume operation and attempt to recover.
Before resetting the WDT (as you say, it should happen in the main routine) you should check as many things as possible, not just that you got to the reset routine. You may wish to confirm that known values such as SFRs have not changed (or re-write them if that does not disturb things). It's a bad idea to depend on power-on default values. It's possible to set flags that indicate error-free completion of important operations- you can make the flags persistent (flag an error and never reset it) then if your WDT reset routine finds that the flag has been set it has the option of performing a complete recovery.
It's good practice to have the WDT reset fenced in by jumps so that you can't run into it by accident (say by executing NOPs in unused memory). Some WDT implementations have a window function that requires a minimum time between resets as well as a maximum, which prevents some unanticipated routine from resetting it accidentally.
As far as the WDT time-out- you want to make it short enough that nothing bad can happen, or the WDT will not be that useful. In the case of a thermal system, a couple seconds might be no problem. In the case of motion control system, milliseconds might be better. Too frequent and you may have problems resetting it frequently enough.
It's better to have a WDT that cannot be disabled from firmware and cannot be set to a too-long timeout (which is almost the same as it being disabled). Usually well-designed processors have some protection against an errant processor doing that sort of thing.
In the case where you have to reset it very frequently, but you have a very slow control loop executing, you can use a flag or a timer to indicate a problem in the slow routine and reset the WDT if it times out. That, essentially, adds a software WDT to the slow routine.
An example of that might be a motion system that does a self-tuning algorithm in the background. You need a very fast WDT to prevent the system from making undesirable movements, but the self-tuning takes longer.