If you wish to save fuel, the of course something's got to give. After all one doesn't get something for nothing. The ECO also affects other systems on the car that consume energy via the engine. For example with the ECO button on, the A/C and its blower are reduced in power. To can demonstrate this to yourself easily. Try this. When the A/C is running and the blower is on, try switching the ECO off. You will notice an increase in the speed of the blower.
I always leave the ECO on. Unless you are used to having pole position on the starting grid then leaving it on is fine. Yes you will notice a lessening of power at low revs. But most people seem to compensate by putting their foot down further, thus consuming more fuel. This of course defeats the whole point of the ECO button.
This is why some people can't notice any significant difference with the ECO button on or off.