I would add thought that it is not a problem, and it is not strange. That is the behaviour that the program was designed with.
Windows Fax and Picture viewer lets you scroll thru the pictures in a working folder (like several other programs do as well).
If you don't want to see the rest of the pictured of the temp folder, either do not scroll thru the picture, use a different program, or save the pictures to a folder of your chosing before opening.
I would like to note also, that no program can scroll thru the pictures in a email message if you don't save them before, so if you get 5 pictures in an email, and open one, you cannot scroll to the next, you will actually have to open the other 4 pictures as well one by one (or save them).
From helpdesk experience, also note that the attachments are saved temporary (as the name of the folder says) and is not a good location to keep. I have a user that lots several files because she was opening the attachments and then savin in the current folder instead to save to another folder.
Adding a extra bit of information too, McAfee (enterprise) for example, will not let you open attachments from the temp folder. You have to save the attachment before, and then open it. The reason for this is to ensure it is scanned with the antivirus, and it is usually a good practice. Make a folder called attachments for example, where you will save the attachments before opening and later you delete them as you don't need them anymore.