In the current design, users can select multiple items with shift + down in multiple selection Listbox. But shift + up doesn't deselect the multiple items. It sounds reasonable to have reversed function for 2 reversed action:
shift + down : select multiple items
shift + up : deselect multiple items

In windows or Mac, you can perform similar selecting and deselecting operations with shift and up / down.

workaround

the attached js