Purpose Accurate co-registration with CT images is a necessity for the implementation of diffusion-weighted (DW) MR imaging for target delineation in radiotherapy (RT) for prostate cancer. Intraprostatic fiducial markers have generally been adopted in RT to improve the image registration precision of the prostate between CT- and MR images. The aim of this study was therefore to evaluate the reconstruction of both gold markers (GMs) and poly markers (PMs) on 1.5T- and 3T-DW MR images as well as cone beam CT (CBCT) scans, using T1-weighted (T1W) MR- and CT images as references. Method Three GMs (1x3mm) and three PMs (0.9x3mm) were fixated into two Agar-gel filled plastic cylinders and placed in an Agar-gel phantom. T1W images (1.5/3T; TE=10/10ms, TR=520/209ms, acquired resolution=2.00/1.85mm/pxl, slice thickness=2/3mm) were obtained in the same frame of reference as DW images (parallel imaging, sense factor=2, 1.5/3T; b-values=600,1000s/mm2, TE=91/70ms, TR=2086-2514/1800ms, acquired resolution=2.31/2.25mm/pxl, slice thickness=4/4mm). A co-registration with CT images (image resolution=0.35mm/pxl, slice thickness=0.7mm) was performed to both T1W- and CBCT images (image resolution=0.88mm/pxl, slice thickness=3mm) based on both the phantom volume and the two embedded plastic cylinders enclosing the GMs and PMs. The marker volumes on both CT- and CBCT images together with the corresponding low intensity area on 1.5T- and 3T-DW images were segmented and evaluated with the Dice similarity coefficient (DSC). In addition, the displacement of the volumes’ center of mass (CM) on DW- and CBCT images relative to the CM obtained on CT was measured. Results On CT- and 1.5T-DW images, there was no difference between the estimated volumes of the PMs and GMs (mean 28±5mm 3 ). On 3T-DW images, the volumes of PMs (50±20mm 3 ) were larger than for the GMs (20±10mm3 ) while on CBCT the GMs (40±10mm3 ) were larger than the PMs (20±10mm3 ). For both the GM and PM volumes there was a moderate overlap between the CT- and CBCT images (DSC=0.51±0.12), whereas there was little overlap between CT vs. both 1.5T- and 3T-DW images (DSC0.2). The CM shift for both GMs and PMs was larger on 1.5T-(6.7±4mm) than on 3T-DW images (3±1mm) whereas the shift was negligible on CBCT (1mm). The CM shift for both marker types was larger when using lateral compared to vertical phase encoding (PE) on 1.5T-DW images while on 3T there was no difference between these directions. Conclusion The reconstructed volume of PMs compared to GMs was smaller on CBCT scans, yet larger on 3T-DW images. On both CT- and 1.5T-DW images the two marker types had identical reconstructed volumes. There was an extended shift for both marker types on 1.5T-DW images in comparison to on 3T. The robustness of marker segmentation when varying 1.5T- and 3T-DW sequence parameters remains to be studied.