An advanced novel Monte Carlo simulation model of the detection process of an optical coherence tomography (OCT) system is presented. For the first time it is shown analytically that the applicability of the incoherent Monte Carlo approach to model the heterodyne detection process of an OCT system is firmly justified. This is obtained by calculating the heterodyne mixing of the reference and sample beams in a plane conjugate to the discontinuity in the sample probed by the system. Using this approach, a novel expression for the OCT signal is derived, which only depends uopon the intensity distribution of the light from the sample and the reference beam. To adequately estimate the intensity distributions, a novel method of modeling a focused Gaussian beam using Monte Carlo simulation is developed. This method is then combined with the derived expression for the OCT signal into a new Monte Carlo model of the OCT signal. The OCT signal from a scattering medium are obtained for several beam and sample geometries using the new Monte Carlo model, and when comparing to results of an analytical model based on the extended Huygens-Fresnel principle excellent agreement is obtained. With the greater flexibility of Monte Carlo simulations, this new model is demonstrated to be excellent as a numerical phantom, i.e., as a substitute for otherwise difficult experiments. Finally, a new model of the signal-to-noise ratio (SNR) of an OCT system with optical amplification of the light reflected from the sample is derived, and discussed. Using this model, the conclusion is reached that an optical amplifier will enable substantial improvement of the SNR for OCT systems dominated by receiver noise. Receiver noise is of practical concern because of the (often) limited irradiance of suitable optical sources for OCT, and high insertion loss of the fast optical delay-line scanners that are necessary for fast imaging. Correspondingly, an increase in penetration depth of about 30-100% is demonstrated for OCT imaging in skin based on results obtained with the new Monte Carlo model. Accordingly, the two new models are demonstrated as valuable tools for future development and optimization of OCT systems to extend the applications of the system in biomedicine.