Google's mission is not "to search", it is "to organize the world's information". The idea that a user's reordering of web pages doesn't affect other users is unlikely to be true for long. If lots of users are saying that result A is better than result B, it won't be long before Google figures out how to data mine the search wiki data to improve search results. Additionally, Google is likely to figure out how to take urls a user manually enters as being relevant and making those more likely to appear in other people's search results. Similarily, Google is likely to figure out how to take feedback from users like you to minimize the intrusiveness of this feature for those who don't want it.