Consumption of a high energy diet, containing high amounts of saturated fat and refined sugar has been associated with impairment of cognitive function in rodents and humans. We sought to contrast the effect of a high fat/cholesterol, low carbohydrate diet and a low fat, high carbohydrate/sucrose diet, relative to a standard low fat, high carbohydrate minipig diet on spatial cognition with regards to working memory and reference memory in 24 male Göttingen minipigs performing in a spatial hole-board discrimination test. We found that both working memory and reference memory were impaired by both diets relative to a standard minipig diet high in carbohydrate, low in fat and sugar. The different diets did not impact levels of brain-derived neurotrophic factor in brain tissue and neither did they affect circulatory inflammation measured by concentrations of C-reactive protein and haptoglobin in serum. However, higher levels of triglycerides were observed for minipigs fed the diets with high fat/cholesterol, low carbohydrate and low fat, high carbohydrate/sucrose compared to minipigs fed a standard minipig diet. This might explain the observed impairments in spatial cognition. These findings suggest that high dietary intake of both fat and sugar may impair spatial cognition which could be relevant for mental functioning in humans.