Digital circuit design is usually done using a HDL language like Verilog or VHDL. This can be simulated on digital level with tools like Modelsim / Questa.
A free alternative for VHDL would be GHDL in combination with GTKwave to view the (digital) waveforms.
These softwares basically run the logic as a program and produce the digital waveforms so you can look at them. Icarus (as stated in the comments) can be used for Verilog.
Digital Asic design is almost identical to FPGA design during the design phases of your logic. You might find a lot more details when looking for free FPGA design techniques.
Analog Design / Full Chip Design
For analog designs and for the final simulations of a digital design, a circuit simulator is needed. Usually, chip development is done in tools like Cadence Virtuoso. You can search online for pictures.
Virtuoso has to be fed with the design kit of the process you want to use, which contains all of the models of the components available in the specific semiconductor technology.
Then you can design your schematic in and simulate it on the schematic level. This is basically very similar to tools like LTSpice etc.
The real difference comes when you go to layout. You can generate a layout of your schematic like in a PCB design software where you can place your transistors etc. and route them together.
There are usually two tools available and integrated that allow you to ensure your layout is working:
- Layout versus Schematic check (LVS): This can detect the structures in your layout as transistor / resistor etc. and check them against the schematic.
- Design Rule Checker (DRC): This checks your layout and ensures you stay within the capablities of your technology regarding: Minimum structure widths, spacing etc.
Aditionally there is usually a way to do a parasitic extraction: This way you can go back from your final layout to the simulation and incorporate the parasistics of your layout into the simulation and do a full simulation of your layout. Depending on your technology and the provided models this is more or less powerful.
Using additional tools like ADS Momentum you can even incorporate 2,5/3D simulations of structures to verify on-chip inductors etc. using electro-magnetic field simulations.
There is also a way to model analog behavior in a description language like VHDL-AMS or Verilog-A. By modelling larger parts of your circuit in an analog description language, you can abstract away a lot of transistors and speed up simulation times. I used to do this for a RAM for example. With a 1 MB SRAM onchip, you need a lot of simulation capability to simulate the millions of trnasistors in the RAM. Instead you can simulate the RAM once and model its outer behavior in a modelling language and therefore massively reduce the node count of your simulation.
Going from Digital to chip design
To design a digital part of a chip, you start with the hardware description as described above and then you "synthesize" the logic from that code. This is very similar to how FPGAs are "programmed". The synthesis generates a "digital schematic" from your description which then can be mapped onto your technology. This mapping knows how a flip-flop, inverter etc. looks in your semiconductor technology. From there usually an automatic place and route is performed which autogenerates the layout.
For this open source tools can be used (I've never done it... So don't know how good it works).
- Using yosys you can synthesize your verilog code.
- QFLow (http://opencircuitdesign.com/qflow/) allows you to do the mapping and routing.
If you configure these tools correctly you're able to generate a digital chip layout from HDL code.
Some software that is also sometimes used for chip development
- Keysight's ADS: A design tool specilaized for high frequency stuff. Usually not for millions of transistor desings but very specific radio frequency signal paths. I used to develop with ADS and then integrate the singal path into Cadence Virtuoso for the complete chip design including logic, power management, etc.
- CST Microwave Studio: A 3D FEM/FDTD electromagnetic solver to simulate 3D structures.
- LtSpice / Spice in general: If you have the right models a totally viable option.
Edit: SW addon:
There is also a tool called KLayout (https://www.klayout.org/) which is able to open and edit GDS files. GDS files are basically chip layout data files. Klayout can also be extended with appropriate design rules etc.
A free "alternative" to Keysight's ADS is Qucs (https://en.wikipedia.org/wiki/Quite_Universal_Circuit_Simulator). It follows similar design flows but somehow lacks usability. IMHO, it's not a real alternative, as it lacks a lot of functionality, but still a good open source software.
One of the biggest problems is: You have to acquire all the necessary configuration / technology information for all of the free tools to be able to get usable results. Most manufacturers will not provide you with this or you have to do a lot of digging yourself. Every technology I've used provided me with a PDK (process design kit) for Cadence Virtuoso.
Is it possible to do an entire chip in open source and/or free software? Yes. It is possible. Know some people from university who did it. But more like a proof of concept to check if it's feasible and can save a lot of money.
My personal experience is that Cadence Virtuoso is the de facto standard and used everywhere. It can be extended with certain tools. But the main work is done in Cadence.
You can simulate Digital designs for free if you use SystemC. The SystemC simulator is freely available from Accellera here: https://www.accellera.org/downloads/standards/systemc
After you verify your design using simulation, Intel provides a free SystemC compiler to convert from SystemC to synthesizable Verilog: https://github.com/intel/systemc-compiler
SystemC supports higher abstractions than Verilog, and it has many benefits, including Object Oriented Digital Design.
UVM-SystemC is freely available from Accellera for Constrained Random Verification.