My feeling from what I have generally read about machine learning approaches is that these adapt better to more complex problems with higher dimensionalities.

Your intuition seems to align with the empirical results in these domains in the last few years. Expert System are in general not used for domains like Computer Vision, but there may be exceptions do this.

Specifically, I am currently trying to make a general comparison (if possible) of expert systems with machine learning systems across dimensions of efficiency and complexity

I will try to provide you with an approach to systematically compare them. You should create a set of tasks $T$ to evaluate your comparison. I will assume your metric to evaluate on is accuracy, but you can of course replace this with any score function of your choice.

Space Complexity

For all tasks implement the expert system and ML model you want to compare. Compare the amount of space they take and average this by the number of tasks to get an estimate of the complexity for each.

$\hat{E}[\mathrm{space}] = \frac{1}{T} \sum_{t\in T} \mathrm{space}(t)$

Time Complexity

Similar to measuring space, but now time each of your model and expert system. You could do so for different functions on each of your tasks such as training or predicting. Training is more difficult, because if you're doing supervised learning for example you don't know when you will stop since more training time might be better. So you should set a time or some criteria of convergence and list them under your assumptions.

$\hat{E}[\mathrm{time}] = \frac{1}{T} \sum_{t\in T} \mathrm{time}(\mathrm{predict}(t))$

Run-time-efficiency

Here is where things get tricky, because there isn't a single unified expression for this that I'm aware of. So I would suggest to evaluate this based on dividing the accuracy by the approximate number of operations. This could be done multiple times and averaged for example if training a neural net for each epoch. To reflect the actual run time in practice more accurately you might also just want to add a time comparison in the form of accuracy divided by runtime on fixed hardware.

All of this provides of course no strong theoretical explanation, but is more of a study to provide some empirical data. If this is done on a large enough scale your results become a lot more meaningful. Scale here means the number of different model types and expert systems and tasks you're testing on.