As some of you have pointed out in the comments on previous posts, there is one not-so-secret way to get useful feedback on student evaluations. The (not really a) secret? Tell your students how important the evaluations are to you.
Perhaps some of you already do this yet your students are so awful and jaded that they still give you unhelpful feedback. Although knowing that I rate as a 4.4/5 for availability is useful, what I really look for are specific comments about teaching style, content, office hours, etc. These comments give me the most talking points for my tenure report.
A few days before I plan to hand out the evals I explain to the class who exactly reads them and what they do with them. Here at PermaU, first the head of the department reads them, then the dean, and then me. If there are any issues (good or bad), I will have a meeting with them, but that doesn't typically happen. I tell the students this, honestly telling them that for right now, I am the only one who will really use them. I explain that I really need honest responses in order to adjust my teaching in future semesters. I especially emphasize that if I am doing something that they like, that they should mention that in the comments so that I keep doing the same thing in future semesters.
Many of my colleagues complain that they get very few responses in the short answer portion of the evaluations, and some say that if they are doing an okay job that their students won't make any comments at all. I think their students just think that evals are only places to report problems not comment on successful practices. By emphasizing to my students that both "bad" and good feedback is useful, I think that students feel that the time they spend filling out the evaluations is well spent. (..and it doesn't hurt that positive feedback is good for my ego!)
After all, if you know that something you are doing doesn't matter, would YOU put time into it?