Your #defines are very confusing. You're not defining a port address at all; you're giving a bitmask for your switch. Also you do not show whether you are putting these port pins in GPIO mode. The default state of PINSEL1 is to have everything in GPIO mode, but you should never make assumptions.
#define SW_MASK 0x3f
#define SW_OFFSET 16
PINSEL1 &= ~(0x00ffffff);
IO1DIR = ~(SW_MASK << SW_OFFSET);
int sw = (IO1PIN >> SW_OFFSET) & SW_MASK;
This is basically the code you have, although with the PINSEL1 code, and arranged to be a little less confusing (to me).
Jobi's right -- you better have a pull-up resistor on P0.20 so that the chip doesn't force these pins into their "trace port" function. See page 70 of the LPC213x user manual.
Take care -- you have put one of the DIP switch switches on P0.20. Either isolate that pin on a reset (a little transistor with an RCD timer on the base which disconnects the switch from P0.20 when you reset the CPU will do the trick), or make sure that you never leave the DIP switch on P0.20 "on" (connected to ground) when resetting. Better yet: just move that switch over to a different I/O and recombine it in code so that your application never knows the difference.
e.g. if you move that switch from P0.20 to P0.22, you can get your "clean" value back thusly:
#define SW_MASK 0x6f
#define SW_OFFSET 16
PINSEL1 &= ~(0x03ffffff);
IO1DIR = ~(SW_MASK << SW_OFFSET);
int sw;
sw = (IO1PIN >> SW_OFFSET) & SW_MASK;
if(sw & (1 << 6)) sw |= (1 << 4);
sw &= ~(1 << 6);
(there are slicker ways to do this, but I'm trying to show you what's going on, not be slick.)
I've altered the mask so that you don't read in P0.20 (SW_MASK change), and that you set P0.22 to GPIO as well (PINSEL1 change). The block that calculates 'sw' will set bit 4 if bit 6 is set (this is P0.20 which was moved to P0.22). It will then "clear out" bit 6, which is now in bit 4's position. The value of your switch will thus be correct.
(most embedded systems don't care for actual bit positions, but you may, which is why I suggested this.)