Wndsworth Road is a very long road. You will find there are restrictions at Lavender Hill end but you should be OK From Queenstown road to Vauxhall. Check the zone times.
Look at the signs. If there are double yellows or reds, then no. If it's a single yellow, then look at the time restrictions on the signs. Usually parking restrictions and parking charges operate on bank holidays regardless.
