updated 07/17/2014 AT 11:00 AM ET
•originally published 07/17/2014 AT 6:15 PM ET
Johnny Depp is making the rounds on the Boston concert circuit.
Just a few weeks after he took the stage to accompany Willie Nelson on guitar, Depp performed with Steven Tyler and the rest of Aerosmith at their show Wednesday night at the Xfinity Center in Mansfield, Massachusetts.
“He played guitar on the encore,” one concertgoer told PEOPLE.
Dressed casually in jeans and a black knit cap, Depp strapped on a guitar to play “Train Kept a Rollin” with the band. His fiancée, Amber Heard, was also rumored to be backstage.
Depp is in Boston to wrap up filming Black Mass, the story of notorious gangster Whitey Bulger. On Monday, he was spotted shooting scenes with costar Sienna Miller, who plays Bulger s longtime girlfriend Catherine Grieg.
The pair walked Miller s character s dog along a golf course at Nashawtuc Country Club in Concord, Massachusetts. Over the weekend, they were spotting filming on Revere Beach just north of Boston.