Buyer said not receive but tracking shows Delivered

I have been seller for a few months only, I got a message from a buyer today saying he has not received the parcel, he said he has been at home all day and no parcel in mailbox. I checked the tracking number, it shows Delivered yesterday. I then replied him and provided the tracking number, told him to check with the delivery carrier with this tracking number and they should be able to trace the parcel. He then replied back that “yes tracking shows Delivered but I haven’t received the parcel!”

Now what do you do in such case? I think either he is a scammer, or he should check with the delivery carrier/post office, and I don’t want to issue refund honestly… but I’m afraid he will leave me the negative feedback or raise claim..

Any suggestion? Will you issue refund? (just treat it as my bad luck this time), or just not reply him? Or tell him again to check with delivery carrier?