For yay, the cache is in

$HOME/.cache/yay/

If the package is there, then: sudo pacman -U ./quarto-cli-1.4.555-1-x86_64.pkg.tar.zst

Then pin the package in /etc/pacman.conf:

# Pacman won't upgrade packages listed in IgnorePkg and members of IgnoreGroup
IgnorePkg   = quarto-cli
#IgnoreGroup =

Then yay -Syu will ignore it as well:

:: Synchronizing package databases...
 endeavouros is up to date
 core is up to date
 extra is up to date
 multilib is up to date
:: Searching AUR for updates...
 -> quarto-cli: ignoring package upgrade (1.4.555-1 => 1.5.52-1)
:: Searching databases for updates...
 there is nothing to do

Refs: How do you downgrade an AUR package? : r/archlinux

For not-AUR, there’s the downgrade command: archlinux-downgrade/downgrade: Downgrade packages in Arch Linux