I actually talked about this with my group and one of the players noted that this actually isn't true.
"Activating some magic items requires a user to do something special, such as holding the item and uttering a command word. The description of each item category or individual item details how an item is activated. Certain items use one or more of the following rules for their activation.
If an item requires an action to activate, that action isn't a function of the Use an Item action, so a feature such as the rogue's Fast Hands can't be used to activate the item." (DMG p. 141)
I find it a bit jarring that it actually calls out Fast Hands not being able to activate magic items that are "Use an Item" actions. Drinking a potion is under Consumables and they seem to be Use an Item actions. I'm not sure what the Fast Hands "Use an Object" is supposed to be then. shrug