The solution seems to be to visit https://huggingface.co/CompVis/stable-diffusion-v1-4 and check a checkbox and click the button to confirm access.

The full error I also got was:

  HTTPError: 403 Client Error: Forbidden for url: https://huggingface.co/api/models/CompVis/stable-diffusion-v1-4/revision/fp16

I visited https://huggingface.co/api/models/CompVis/stable-diffusion-v... and saw

  {"error":"Access to model CompVis/stable-diffusion-v1-4 is restricted and you are not in the authorized list. Visit https://huggingface.co/CompVis/stable-diffusion-v1-4 to ask for access."}

Eventually I focused and realized I need to visit that URL to solve the issue. Hope this helps.