roofAI 🏛️: serving — 6

Next, in roofAI 🏛️,

cloudwatch help

cloudwatch browse endpoint

roof inference invoke \
  profile=VALIDATION - \
  $(@ref roofAI_ingest_AIRS_v2) \
  $(@timestamp)

@git review

import boto3
from sagemaker import image_uris

image_uris.retrieve("xgboost", boto3.Session().region_name, "0.90-1")

rerun https://github.com/kamangir/roofAI/tree/main/roofAI/inference with pytorch ▶️ update doc,

@select $(@ref roofAI_semseg_model_AIRS_o2)
roofAI inference create model .

botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreateModel operation: Region of “763104351884.dkr.ecr.us-east-2.amazonaws.com/pytorch-inference:2.1.0-gpu-py310-cu118-ubuntu20.04-ec2” does not match expected region of ca-central-1. Cross region ECR image pulls are not allowed.

aws region fixes.

@select $(@ref roofAI_semseg_model_AIRS_o2)
roofAI inference create model .

roofAI inference create endpoint_config,suffix=pytorch .

roofAI inference create endpoint,config_suffix=pytorch,suffix=pytorch .

cloudwatch browse endpoint

⏬

roof inference invoke \
  profile=VALIDATION - \
  $(@ref roofAI_ingest_AIRS_v2)

roof inference describe endpoint \
  endpoint-model-2023-12-03-11-24-39-75649-pytorch

⏳

roof inference describe endpoint \
  $(roofAI_inference_default_endpoint)

Image size 13474926778 is greater than supported size 10737418240

▶️ https://repost.aws/questions/QU90699ONgQD2t2HUKzm9AUA/failure-reason-image-size-12704675783-is-greater-than-supported-size-10737418240-when-creating-serverless-endpoint-in-sagemaker

You need a smaller container image 🤨. Also, take into consideration that at the moment SageMaker serverless endpoints do not support GPU acceleration 🤨

updating to the CPU model.

function runme() {
    local endpoint_config_name
    for endpoint_config_name in \
    config-model-2023-12-03-11-24-39-75649-pytorch \
    config-model-2023-12-03-11-24-39-75649-v1 \
    config-model-2023-12-02-19-58-34-09697-ysbKRprW \
    config-model-2023-12-02-19-58-34-09697-v2 \
    config-model-2023-12-02-19-58-34-09697-gF1bLja1 \
    config-model-2023-12-02-19-58-34-09697-v1 \
    config-model-2023-12-03-11-24-39-75649-Wnmubt9u \
    model-2023-12-02-19-58-34-09697-5PcsJqto \
    model-2023-12-02-19-58-34-09697-17yYwi4E \
    model-2023-12-03-11-24-39-75649-pEoSptOf ; do
        roof inference delete endpoint_config $endpoint_config_name
    done
}

runme "$@"

function runme() {
    local endpoint_name
    for endpoint_name in \
        model-2023-12-03-11-24-39-75649-Od0CqqOz \
        model-2023-12-03-11-24-39-75649-lLhMVWru \
        model-2023-12-03-11-24-39-75649-GkklwnYc \
        model-2023-12-03-11-24-39-75649-OPVtS9KM \
        model-2023-12-02-19-58-34-09697-wSl6BFZb \
        model-2023-12-03-11-24-39-75649-rM9cAqWB \
        model-2023-12-03-11-24-39-75649-jiobxt4Q \
        model-2023-12-03-11-24-39-75649; do
            roof inference delete endpoint $endpoint_name
    done    
}

runme "$@"

@select $(@ref roofAI_semseg_model_AIRS_o2)
roofAI inference create model .
roofAI inference create endpoint_config,suffix=pytorch .
roofAI inference create endpoint,config_suffix=pytorch,suffix=pytorch .

roof inference describe endpoint \
  $(roofAI_inference_default_endpoint)

FileNotFoundError: [Errno 2] No such file or directory: ‘serve’

Next: retry.