Yes, stuff like this matters. It matters because the programmers at Google designed the search engine to recognize and search for programming syntax and the underscore is common in programming languages.
Google is still a dumb robot, it is getting better at understanding the intricacies of language, but it is still just a program that designed to take input and make a specific output. Which isn’t great for words, characters, or symbols that have multiple meanings.
Don’t bother changing your URL structure if you are already using underscores.
From a developer’s perspective it’s easier not to hit the Shift Key for every URL. Your users won’t like it either and that adds up to a lot of bad user experience at scale.
Basically, underscores are for programmers and machines, and have a very specific meaning. Dashes or hyphens are for people. It’s not very surprising considering that Google has been pretty consistent about this.
Here’s the earlier video from 2009 that Matt Cutts referenced in the previous video.
When in doubt, do the simplest easiest thing, focus on usability, and don’t over-think your SEO.