Search code examples
pythonamazon-web-servicesscalaaws-glue

aws glue version 0.9 python and scala scripts testing


We will be working on aws glue 0.9 version to 4.0 upgrade.As part of the analysis,we were checking the changes to be done. For testing purpose we have creatred some sample aws glue 0.9 python and scala programs.But the challenging part here was,since 0.9 version is decommisioned.we are not able to test the 0.9 scripts which we have

Below is from the aws documentation:

AWS Glue no longer accepts a null parameter. If you pass 0.9 or 1.0 in the parameter you encounter the error "Glue Version 0.9 (or) 1.0 is not supported."

As of now we are just assuming that the scripts which were created as part of testing ( 0.9 version) are correct but its not the right way. Could you please let us know if there is any way where we can test the aws glue 0.9 python and scala scripts.


Solution

  • There is no way to test 0.9 scripts that I'm aware of. You might look into developing locally, however please note: local development does not always guarantee that the script would work on AWS.

    That being said, there are very significant changes between versions (methods no longer supported, performance issues, parallelisation issues) - and at least 3 breaking upgrades (0 -> 1, 2 -> 3, 3 -> 4), so depending on work involved you might want to start rewriting the scripts from scratch.