I am thinking of creating an AI video generation app that incorporates Hotshot.
However, the inference.py cloned directly from github works, but when I import the modified inference.py in the app I am building, I get the following error.
ValueError: unet/hotshot_xl.py as defined in
model_index.json does not exist in hotshotco/Hotshot-XL and is not a module in 'diffusers/pipelines'.
Modified inference.py
def main(pretrained_path='hotshotco/Hotshot-XL',
xformers=False,
spatial_unet_base=None,
lora=None,
output='output_scuba.gif',
steps=30,
prompt='Sasquatch scuba diving, anime style',
negative_prompt='blurry',
seed=455,
width=672,
height=384,
target_width=512,
target_height=512,
og_width=1920,
og_height=1080,
video_length=8,
video_duration=1000,
low_vram_mode=False,
scheduler='EulerAncestralDiscreteScheduler',
control_type=None,
controlnet_conditioning_scale=0.7,
control_guidance_start=0.0,
control_guidance_end=1.0,
gif=None,
precision='f16',
autocast=None):
args = argparse.Namespace(
pretrained_path=pretrained_path,
xformers=xformers,
spatial_unet_base=spatial_unet_base,
lora=lora,
output=output,
steps=steps,
prompt=prompt,
negative_prompt=negative_prompt,
seed=seed,
width=width,
height=height,
target_width=target_width,
target_height=target_height,
og_width=og_width,
og_height=og_height,
video_length=video_length,
video_duration=video_duration,
low_vram_mode=low_vram_mode,
scheduler=scheduler,
control_type=control_type,
controlnet_conditioning_scale=controlnet_conditioning_scale,
control_guidance_start=control_guidance_start,
control_guidance_end=control_guidance_end,
gif=gif,
precision=precision,
autocast=autocast
)
# The following is the same as inference.py
Some of the Python code for my app
if generate_value == 4:
main(pretrained_path='hotshotco/Hotshot-XL',
xformers=False,
spatial_unet_base=None,
lora=None,
output=filename_vid,
steps=steps,
prompt=prompt,
negative_prompt=negative_prompt,
seed=455,
width=width,
height=height,
target_width=width,
target_height=height,
og_width=width,
og_height=height,
video_length=video_length,
video_duration=1000,
low_vram_mode=False,
scheduler='EulerAncestralDiscreteScheduler',
control_type=None,
controlnet_conditioning_scale=0.7,
control_guidance_start=0.0,
control_guidance_end=1.0,
gif=None,
precision=precision,
autocast=None
)
(These were all translated from Japanese by DeepL)
I was hoping to incorporate the main function with prompt, negative prompt, length of video to generate, precision, width and height as arguments in my app and then create a gif based on it, but I got an error.
I found the cause of this problem. It is the difference between running in py or python.
# Success example:
python main.py
# Failure example:
py main.py.
```