I've just downloaded the Action Bar Icon Pack from https://developer.android.com/design/downloads/index.html, and I've not been able to figure out a way to "install" these icons in Android Studio... Should I copy&paste it to the res/drawable folder, one by one? Or is there some dir where I could unzip them all?
The Action Bar Icon Pack is similar to the a normal drawable resource that is used in the any project. The purpose of this is for designer to have a guideline to develop new icons or customize existing icons using tools like Illustrator
To answer your question -
Use Existing icons - copy/past the icons to desired res/drawable folder as you wish to use it.
Create/Edit icons - Import the icons to your Image Editor's(eg. Illustrator) to edit/create icons.