DWI PHILPOT^Kathleen^Margaret^Mrs^

Inference FAILED: infer subprocess failed rc=1
<frozen importlib._bootstrap_external>:1328: FutureWarning: The cuda.cudart module is deprecated and will be removed in a future release, please switch to use the cuda.bindings.runtime module instead.
[s3] downloaded 428 objects from s3://ahi-incoming/batches/20260524T085604Z_1_3_6_1_4_1_14301_77_4_45748008_1/
[select] DWI selected desc='RESOLVE_4SCAN_TRACE_TRA_P2_150_TRACEW' b=0.0 files=54 slices=54
[select] Selected SeriesInstanceUID=1.3.12.2.1107.5.2.51.182654.30000023102312133767100000006 SeriesNumber=5001 instances=54
[select] route=dwi model=dwi_segmentation
[select] DWI b-value filtering kept 27/54 slices at b~1000.0 (tol=120.0)
[convert] DICOM -> NIfTI: /srv/infer-work/tmprhsxbg2t/output/input_image.nii.gz
Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
        (self._dns_host, self.port),
    ...<2 lines>...
        socket_options=self.socket_options,
    )
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    response = self._make_request(
        conn,
    ...<10 lines>...
        **response_kw,
    )
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/urllib3/connectionpool.py", line 493, in _make_request
    conn.request(
    ~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<6 lines>...
        enforce_content_length=enforce_content_length,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/urllib3/connection.py", line 445, in request
    self.endheaders()
    ~~~~~~~~~~~~~~~^^
  File "/home/ec2-user/miniconda3/lib/python3.13/http/client.py", line 1333, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/lib/python3.13/http/client.py", line 1093, in _send_output
    self.send(msg)
    ~~~~~~~~~^^^^^
  File "/home/ec2-user/miniconda3/lib/python3.13/http/client.py", line 1037, in send
    self.connect()
    ~~~~~~~~~~~~^^
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/urllib3/connection.py", line 276, in connect
    self.sock = self._new_conn()
                ~~~~~~~~~~~~~~^^
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
        self, f"Failed to establish a new connection: {e}"
    ) from e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f8b4fe6ba10>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
        method=request.method,
    ...<9 lines>...
        chunked=chunked,
    )
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/urllib3/connectionpool.py", line 841, in urlopen
    retries = retries.increment(
        method, url, error=new_e, _pool=self, _stacktrace=sys.exc_info()[2]
    )
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=8000): Max retries exceeded with url: /infer/dwi_segmentation?output=image (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8b4fe6ba10>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/AHIR/s3_to_monai_infer.py", line 1708, in <module>
    main()
    ~~~~^^
  File "/opt/AHIR/s3_to_monai_infer.py", line 1516, in main
    infer_txt = run_monai_inference(dict(infer_params))
  File "/opt/AHIR/s3_to_monai_infer.py", line 1494, in run_monai_inference
    monailabel_infer_file(
    ~~~~~~~~~~~~~~~~~~~~~^
        monai_url=monai_url,
        ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
        infer_params=params,
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/AHIR/s3_to_monai_infer.py", line 513, in monailabel_infer_file
    r = requests.post(f"{monai_url}/infer/{model}", params=params, data=data, files=files, timeout=1800)
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/home/ec2-user/miniconda3/lib/python3.13/site-packages/requests/adapters.py", line 700, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=8000): Max retries exceeded with url: /infer/dwi_segmentation?output=image (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f8b4fe6ba10>: Failed to establish a new connection: [Errno 111] Connection refused'))