Most problems in engineering and natural sciences involve parametric equations in which the parameters are not known exactly due to measurement errors, lack of measurement data, or even intrinsic variability. In such problems, one objective is to compute point or aggregate values, called "quantities of interest". In such a setting, the parametric equations must be accurately solved for multiple values of the parameters to explore the dependence of the quantities of interest on these parameters, using various so-called "sampling methods". In almost all cases, the parametric equations cannot be solved exactly and suitable numerical discretization methods are required. The high computational complexity of these numerical methods coupled with the fact that the parametric equations must be solved for multiple values of the parameters make UQ problems computationally intensive, particularly when the dimensionality of the underlying problem and/or the parameter space is high. This thesis includes five published articles and one soon-to-published one. These articles are concerned with optimizing existing sampling methods, namely Multilevel Monte Carlo, and developing novel methods for high dimensional problems, namely Multi-index Monte Carlo and Multi-index Stochastic Collocation. Assuming sufficient regularity of the underlying problem, the order of the computational complexity of these novel methods is, at worst up to a logarithmic factor, independent of the dimensionality of the problem. The articles also explore different applications, including an elliptic partial differential equation that models the flow of a fluid through a porous medium with random permeability and a stochastic particle system that models a system of coupled oscillators.