Any extra piece of glass will degrade your lens, just good filter doesn't affect as much. Personally, I have never used protective filter unless in really harsh condition.
There is no need to replace the entire lens if you should accidentally scratch the front element. It depends on the lens, but it may cost as little as $250 to change the front element on a $1500 lens.
For me it just seems crazy to spend up to $100 for a piece of glass to protect a $250 piece of glass. Especially since "Protection" filters are notoriously fragile and prone to scratches. Over time the filter will need to be replaced as they don't stand up to multiple cleanings as well as the front element of a lens.
All those photos of smashed filters "Protecting" their lenses are really just evidence of how poor a job they do at protection. Often the slightest bump will shatter the filter into tiny pieces that get everywhere and are a pain to clean up afterwords.
"... but it may cost as little as $250 ..."
And it "may" cost quite a bit more! Plus not to mention the hassle it is to box it and ship it.
Not everybody lives next doot to a repair facility. Now we are talking time down, too.
Sometimes one of my "protection" filters has moved from lens to lens, so your $100 filter now costs me $50 bucks or less.
Why do you people not understand a filter is 'removable'? It comes off as easily as it went on!
If a situation warrants it, take it off. Otherwise, it is good protection even if only slight protection.
The 100-400L has 17 elements inside that the light has to pass through to get to the sensor. The majority of these are precisely curved to allow proper redirection of light while minimizing signal loss. Would you have not bought the lens if it happened to have 18 elements? A simply flat piece of glass is trivial in manufacture and design compared to the elements inside the lens. Aside from ghosting issues, I haven't seen any convincing evidence that UV filters noticeably degrade the quality of the final photo. Using a filter is a personal choice. To me, it's worth it, considering low cost and negligible IQ loss.
I think the OP was wondering specifically about using filters with the Canon 100-400 as there are lots of stories all over the internet about people complaining about softness with the 100-400 that went away once they took the "protection" filter off.
To answer his question, yes, according to the many stories I have read, ANY filter on a 100-400 wil cause softness and/or focus problems.