This is part of a series of questions on the Gemara in Bechoros 8b-9a. Below is a summary of the relevant parts of the Gemara. After being challenged by Caesar, R' Yehoshua went to Athens to defeat the Athenian Elders in a battle of wits.
In the process of their debate, the following cases are brought up.
The Elders challenged R' Yehoshua to build a house in midair. R' Yehoshua said the Shem of Hashem and began levitating. "Bring me some bricks and mortar," he said, "and I'll do it." (The story doesn't make clear when, exactly, he dropped back down to Earth, but presumably he does eventually.)
Further down, the Elders ask him to bring a well from the desert into the city. "Bring me a rope of bran," he said, "and I'll do it."
The Elders finally ask R' Yehoshua to sew together a broken millstone. "Bring me threads of stone," he said, "and I'll do it."
In respect to the second case, with the well, Rashi says that "if you can fulfill my request (which you can't), I will fulfill yours."
Since R' Yehoshua uses a similar language in all three cases, I would assume that he meant the same by all of them. So how did he answer their challenge by saying that he won't? He's not fulfilling their challenge or saying that it's possible; all he's saying is that he'll do it when the impossible happens, which is essentially saying no in a dignified manner.