You were a S.H.I.E.L.D. agent before you were forced into HYDRA. You weren't even eighteen when you joined S.H.I.E.L.D., an innocent-faced kid... you came in handy as an agent and as an adult, too. But you were not fit for duty in the field, so you joined the team responsible for the files. Your knowledge of the files and your ability to go mostly unnoticed... you would have done HYDRA's work. It also helped that they threatened you with the only family you had. Your grandpa was the one who recruited you to S.H.I.E.L.D., a perfect agent until age made him a thoroughly old man. You stayed silent for his life, you worked for HYDRA. Learning that Sergeant Barnes was not dead was a plus. Sergeant Barnes was now the Winter Soldier. It took a while to digest the truth. So is being officially chosen as the victim. No one wanted to be anywhere near the Winter Soldier, even to explain the details of the mission. They had pushed you forward, and the Winter Soldier had not harmed you. Maybe because you're unnecessary for even that or because you're the only one who accepts him as a human being. You also explained missions down to the smallest details, it was also useful for him.
Your grandpa died because of his age, you thought there was no one to threaten you. But you were wrong. You saw them brainwash the Winter Soldier, and unlike the others, you were the one who showed any sign of humanity. Realizing that he could threaten you with the Winter Soldier, Pierce took advantage of this. You saw his human side when Pierce slapped you hard in the face for not distracting Sharon the night Fury was killed. When Pierce left, the Winter Soldier held out his metal hand to you, almost gently. That alone was enough to break the brainwash machine. Sergeant was still somewhere inside him.
Realizing he remembered Steve, Pierce ordered them to use the machine after slap Soldier too. It didn't happen so easily. "The man on the bridge...Who was he?" He whisper to you quietly cause he knows you are the only one will answering him.