hadoop/spark

[skiptoniam]

This tutorial seems to be well out of date. I managed to get it running by folking the elasticluster repo and making a lot of changes. I seem to have a few issues actually running the hadoop cluster, which I'm still working through, I expect some changes to the ansible playbooks might have cause issue with the hadoop/spark cluster.

[ramson33]

Hi there,

Thanks for letting us know, and sorry there were issues with the tutorial!

I will have a look through and make fixes where necessary.

Kind regards,
Sonia

[skiptoniam]

All good. I also had to run elasticluster from a python3.7 virtual environment. It didn't work from 3.8.

[andybotting]

hey @skiptoniam We've had a couple of tickets come through recently about this. I've been trying to make this work with at least Python 3.10 and I've fixed up a few things.

We're trying to work out if we can continue to support this tutorial, as it does feel like upstream aren't really maintaining it.

Maybe with the work you've done here and my work fixing some of the newer Python issues, we can keep it going. Are you going to be using Elasticluster longish term?