Raising your calories to say 2900 and still lifting progressively should help you gain, work your way up slowly.
What do you mean by gains seem to fade away, I would say you are expecting dramatic results, quality gains take time, years, even with gear.
Eating 5 times a day is the minimum for me haha... I eat 6 days every day 100% Although I eat up to 4000 calories a day and I'm 230-240lbs and I need to increase my calories.
Hard to tell without pictures your current conditioning and size.
Also being more specific on nutrition matters, just saying you are eating aproximately some foods is not the same as saying I eat this specific type of meal. Just saying whole food is very broad. I eat chicken breast, salmon, rice, eggs, these are specifics.