1) You may try Assembla which is pretty much you'll ever need. You as admin can filter tabs your client can see. For example, leaving him Tasks tab only where he can make tasks for you, make comments on each, etc.
2) The other one is to try Trello which is visually richer. Yet again, he may not like drag&drop things, popups etc. So maybe Assembla may be more Win95 in style :) and they may like it more.
Both are free to use and located in cloud.
3) Third option is to use Google Docs where you set a public Excel doc where you both can enter tasks. Some of my clients prefer Excel, especially older ones.
4) Another option which is liked my current older client (55 yrs) is Basecamp room. He can write comments there, post images and docs, all is drag&drop. He is not confused. Good thing is that Basecamp offers you 3 months for free. If you cannot afford Basecamp and project takes more than 3 months, then before 3 months-trial ends, export all the content and open a new free account. In a long run, this is pain and eventually you'll start with a paid account.
But you must be clear with the client that they should put effort in the project as well since you are not coding it alone but together with him.