I am planning on using CAN bus in an industrial automation application, however I'm having a hard time deciding how to implement the network layer. Does anyone here have any recommendations for a higher layer protocol to use over CAN that provides guidelines for how to assign message IDs and how to format data packets? Should I develop my own proprietary network layer or implement an existing standard like CANopen?
You will definitely want to choose a standard network layer. I've seen some cases where people have rolled their own, and frequently the protocol is inefficient, and in the end it's a waste of time. A proven network layer provides basic functionality to solve most of the problems you'll likely face in networking with an embedded device, and that will save you precious time and money in the end.
I have familiarity with CANopen, so I'll put a bit of an explanation in here and it might help you decide if that's the way you want to go. If it doesn't fit what you need to do, I'd look at something else.
CANopen is very data centric. All CANopen slave nodes (things you control from a master node, like sensors or motor drivers) provide a dictionary that contains all configuration, control and data-acquisition data you need to read/write to get your job done. You can then read/write these objects individually via the SDO (service data object) mechanism, or read/write these objects periodically in timed CAN message transmissions via the PDO (producer data object) mechanism. Startup, shutdown, and reset, and heartbeat/node-malfunction detection are provided from NMT services, as well as a node ID assignment service (if you wish to use it). To implement a device using an embedded CANOpen library, you basically setup an object dictionary, setup PDOs to periodically send/receive data updates, and write user code that behaves based on those data entries. There are other things CANopen provides, but that is the bulk of it. Read this book for more info.
I write code to access the CAN bus on GM cars, which uses a mostly proprietary network layer. Have you looked into OpenCAN? I would choose an open standard such as CANOpen or even OpenCAN. It really depends on the level of control and flexibility you want in your automation environment.
There have been great answers. I do not know what micro you use, but here are links to some info from microchip.
They each have code and a basic datasheet letting you know what you need to implement them.