Just to let you know that you are not alone in that sense of panic when hearing more competent learners speak. I'm a bit all over the shop with my speaking abilities. I've not been doing badly recently and in a few 1-to-1 Skypes I've done recently I feel I've been doing quite well. Then the other evening I did a late-night chat with 2 of the more competent speakers via the new Slack website - not helped by the fact that I was staring into a blank screen with moving circles and one of them was on a phone with the sound fluctuating wildly - and I more-or-less went to pieces. They were not asking me anything particularly difficult but I just wasn't picking it up, leading to a loss of confidence and a struggle even to be able to say basic things like 'dw i'n' and stuff! After that call I was almost on the point of jacking the whole thing in - but by the morning common sense prevailed once again, and "Don't be so silly".
It's a funny business. When I was very much younger I studied French & German to A Level - this was the old days when 75% of it was grammar and literature and I was on the point of becoming fluent. But that's now - I hesitate to say it - just over 50 years ago, and now well into retirement this language stuff doesn't come easy. Plus all the mixed emotions about my Welsh background etc. But the only way to look at it is that SSIW has given me the opportunity to come THIS far - which I never thought I'd ever have (living as I do on Ynys Wyth - the Isle of Wight). And that's what we must do, as you say, constantly reminding ourselves just how far we have already come.
I see you're doing the 6-month intensive. I haven't signed up as I had the opportunity to do the 5-day intensive last October which it's based on I think, and that's been a great boost. But it still doesn't solve the fluctuating confidence-thing. Anyway, just to let you know that many of us are struggling in various ways.