Support of venv with pyspark #65
Description
Building and packaging the virtual environment with an EMR on EKS container image should allow a user to use spark.archives conf.
Note, one should not try to build the archive locally, because there is an issue with how symlink work and would end up throwing an error that the file (python binaries) do not exist. This is because the sym link point to the location of the binaries on the local machine, which are not the same as in the emr on eks container image.