AWS Glue for Spark のジョブから AWS CodeArtifact を経由して PyPI のライブラリをインストールする

# ローカルで指定した時
$ python3 -m pip install --no-cache --no-cache-dir -U --index-url INDEX_URL art
Looking in indexes: https://aws:****@aws-glue-package-installing-demo-151671725617.d.codeartifact.ap-northeast-1.amazonaws.com/pypi/aws-glue-package-installing-demo/simple
Collecting art
  Downloading https://aws-glue-package-installing-demo-151671725617.d.codeartifact.ap-northeast-1.amazonaws.com/pypi/aws-glue-package-installing-demo/simple/art/6.5/art-6.5-py3-none-any.whl (610 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 610.4/610.4 kB 2.5 MB/s eta 0:00:00
Installing collected packages: art
Successfully installed art-6.5

[notice] A new release of pip is available: 25.0 -> 25.1
[notice] To update, run: pip install --upgrade pip

ERROR	2025-04-27T08:15:26,631	19580	com.amazonaws.services.glue.utils.concurrent.PythonModuleTask	[pool-4-thread-1]	36	Setting Up Virtual Env and Application Python Modules encountered exception. Exception: Installation of Additional Python Modules failed, please see the logs for more details
com.amazonaws.services.glue.PythonModuleInstallException: Installation of Additional Python Modules failed, please see the logs for more details
	at com.amazonaws.services.glue.PythonModuleInstaller.installCustomerPythonModules(PythonModuleInstaller.scala:139) ~[0rJH7L-aws-glue-di-package-5.0.400.jar:?]
	at com.amazonaws.services.glue.utils.concurrent.PythonModuleTask.call(PythonModuleTask.scala:31) ~[0rJH7L-aws-glue-di-package-5.0.400.jar:?]
	at com.amazonaws.services.glue.utils.concurrent.PythonModuleTask.call(PythonModuleTask.scala:14) ~[0rJH7L-aws-glue-di-package-5.0.400.jar:?]
	at com.amazonaws.services.glue.utils.concurrent.GlueConcurrentAsyncProcessingService.$anonfun$processTaskAndReturnFuture$1(GlueConcurrentAsyncProcessingService.scala:40) ~[0rJH7L-aws-glue-di-package-5.0.400.jar:?]
	at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659) ~[scala-library-2.12.18.jar:?]
	at scala.util.Success.$anonfun$map$1(Try.scala:255) ~[scala-library-2.12.18.jar:?]
	at scala.util.Success.map(Try.scala:213) ~[scala-library-2.12.18.jar:?]
	at scala.concurrent.Future.$anonfun$map$1(Future.scala:292) ~[scala-library-2.12.18.jar:?]
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33) ~[scala-library-2.12.18.jar:?]
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33) ~[scala-library-2.12.18.jar:?]
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64) [scala-library-2.12.18.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: java.lang.RuntimeException
	... 14 more
INFO	2025-04-27T08:15:26,630	19579	com.amazonaws.services.glue.PythonModuleInstaller	[pool-4-thread-1]	183	STDOUT Stream: additionalOptions -r --index-url https://aws:****@aws-glue-package-installing-demo-151671725617.d.codeartifact.ap-northeast-1.amazonaws.com/pypi/aws-glue-package-installing-demo/simple --trusted-host aws-glue-package-installing-demo-151671725617.d.codeartifact.ap-northeast-1.amazonaws.com, additionalPythonModules /tmp/glue-job-7009979284799843628/requirements.txt, virtualEnv /tmp/glue_venv Defaulting to user installation because normal site-packages is not writeable
INFO	2025-04-27T08:15:26,631	19580	com.amazonaws.services.glue.PythonModuleInstaller	[pool-4-thread-1]	184	STDError Stream: ERROR: Invalid requirement: '/tmp/glue-job-7009979284799843628/requirements.txt'Hint: It looks like a path. The path does exist. The argument you provided (/tmp/glue-job-7009979284799843628/requirements.txt) appears to be a requirements file. If that is the case, use the '-r' flag to install the packages specified within it.
INFO	2025-04-27T08:15:26,630	19579	com.amazonaws.services.glue.PythonModuleInstaller	[pool-4-thread-1]	182	exitCode Code: 1