Generally speaking, overdrive (O/D) is the highest gear in the transmission. On most cars the automatic transmission has 3 speeds and overdrive (fourth speed). Overdrive allows the engine to have less RPM with higher speed in order to have better fuel efficiency. When you switch it on, you allow the transmission to shift into overdrive mode after the certain speed is reached (usually 30-40 mph depending on the load). When it's off , you limit transmission shifting by third speed.
In normal driving conditions the overdrive should always be on. You may need to switch it off if you drive in mountain areas. (The automatic transmission automatically shifts from O/D to the 3rd gear when it feels more uphill or towing a trailer, the transmission can not decide to stay in O/D or to shift into 3rd gear and it starts to shift back and forth. That is the time you may switch it off and help the transmission to decide)
You also may need to switch it off when you want to slow down using the engine braking, for example, driving down hill. For more details check your owner's manual.