This kind of question normally comes up in a circuit theory class, where we can pretend that things like ideal op-amps and ideal diodes exist. Since you haven't said otherwise, I'm going to assume that's the context of your question.

While it won't hurt you to learn what a window comparator (mentioned by several people in the comments) is, that's likely not the solution your instructor is looking for.

Something more like this will likely satisfy your instructor:

^{simulate this circuit – Schematic created using CircuitLab}

D2 and D3 are zener diodes. Depending on where you are in your course, you might be expected to place ideal voltage sources in these locations instead of zener diodes.

D1 and D4 are ideal diodes (0 forward voltage). They're there to prevent the zeners from conducting from anode-to-cathode (or to prevent current the "wrong way" through if you've substituted voltage sources for the zeners).

OA1 is an ideal op-amp. In particular, this op-amp has no limits on input or output voltage due to its finite supply voltage.

You can see that in this solution, you don't do anything special to supress the output when the input voltage is near zero. You simply arrange that no output is produced until the input voltage exceeds some threshold in the negative or positive direction.

This kind of question normally comes up in a circuit theory class, where we can pretend that things like ideal op-amps and ideal diodes exist. Since you haven't said otherwise, I'm going to assume that's the context of your question.

While it won't hurt you to learn what a window comparator (mentioned by several people in the comments) is, that's likely not the solution your instructor is looking for.

Something more like this can be a building block in an answer that will likely satisfy your instructor:

^{simulate this circuit – Schematic created using CircuitLab}

D2 and D3 are zener diodes. Depending on where you are in your course, you might be expected to place ideal voltage sources in these locations instead of zener diodes.

D1 and D4 are ideal diodes (0 forward voltage). They're there to prevent the zeners from conducting from anode-to-cathode (or to prevent current the "wrong way" through if you've substituted voltage sources for the zeners).

OA1 is an ideal op-amp. In particular, this op-amp has no limits on input or output voltage due to its finite supply voltage.

You can see that in this solution, you don't do anything special to supress the output when the input voltage is near zero. You simply arrange that no output is produced until the input voltage exceeds some threshold in the negative or positive direction.