If one is going full anal on security, the device would never have access to the internet.
The next step up/down would be the device has a dedicated mission and is only brought online to perform a single task and then powered off again.
For the 24/7 devices, all sensitive data needs to be encrypted and stored locally and accessed on a on-demand basis.