Have to wait and see, but at present, I don't think it is going to be the issue perceived. They already exist and are in quite a few cars. The data that they capture can only be accessed and downloaded by the car manufacturer. Also, there will be an immediate challenge if and when passed, pertaining to whom the box belongs. This issue has been to court before and the car manufacturers have in the past, always won. Its their box, despite the fact you, the owner, purchased the car. Access also requires equipment and software only available to the manufacturer. The car manufacturers have been and are reluctant to release the information unless there is a very dire need, such as resolution of a fatality accident, often times which involves the question of possible homicide. Additionally, the data accessed and released is restricted to just the information deemed necessary and warranted relevant resolution of the issue.
I don't see big brother at this specific time, checking our driving habits unless something pursuant to what is addressed above, occurs. And, accessing the data is very, very expensive. I know. I had it done in a fatal accident in Ohio one time, where a question of homicide was paramount to the investigation. The spouse that was a suspect, attempted to block access to the black box data by claiming since he owned the car, he owned the black box. The court ruled otherwise. It still required getting a search warrant, and GM would only release data specific to the issue that was being investigated.
But, I agree, given the direction the government is headed, there is cause to raise eyebrows. They are seeking, and are expected to get approval for flying spy drones over US cities in the very near future. So it is not unreasonable to envision the modern version of 1984 being right around the corner.