It isn't possible to till soil without killing the animals that live in the soil, and taking away habitat and resources from animals that would have lived on that land.
I doubt it's possible to grow crops without killing insects, rodents, birds, and other animals that will try to eat those crops before they get to your table.
If vegetarians think it's ok to kill these animals but not larger animals that are used for their meat then we've already established a hierarchy.
If that's the case I don't think it's unreasonable to put whales at the top of that hierarchy.
Whatever you do to survive you are going to be indirectly involved in killing, this is just part of the suffering inherent in life. As I understand it the Buddha knew that which is why he was more concerned about his disciples not being directly involved as well.