This paper describes the ADAPT Centre submissions to WAT 2020 for the English-to-Odia translation task. We present the approaches that we followed to try to build competitive machine translation (MT) systems for English-to-Odia. Our approaches include monolingual data selection for creating synthetic data and identifying optimal sets of hyperparameters for Transformer in a low-resource scenario. Our best MT system produces 4.96 BLEU points on the evaluation test set in the English-to-Odia translation task