This will likely be the case for legacy responses. Responses previously copied the question and stored it as part of the response, but responses now (as of yesterday) will just reference the question ID instead of deleting the original question. The downside of this is every response before this security change will not work the same as “real” responses, since their original question is not available anymore. Pinning should work for newer responses, though. Please let me know if you're still unable to pin a new response!