UDEV

To access devices connected to USB via adb or fastboot on linux, you need to configure udev rules. For detailed information about udev, take a look at udev - Arch Linux from the ArchLinux wiki (note that UDEV rules are not exclusive to Arch; udev is used in numerous distributions). You can follow the method below to add rules for any user that is part of the plugdev group, or you can follow Google Android's instructions to add rules only for your username.

Follow these steps to setup the correct rules:

1. Verify your username is included in the plugdev group. Type
   groups
   from a terminal and look for plugdev in the listed groups. If you do not see plugdev listed, you can add your username to the group with:
   sudo gpasswd -a username plugdev
   where username should be replaced with your linux username.
2. Copy the set of rules listed below these steps to a text file and save it as /etc/udev/rules.d/51-android.rules. You will need sudo/su to write to that directory. So, for instance:
   sudo nano /etc/udev/rules.d/51-android.rules
   These rules cover all vendors listed by Google. Optionally, you can add just the vendor corresponding to the device(s) you plan to connect to your computer.
3. Restart your computer and then test plugging in your device to the computer with USB debugging enabled.

#Acer
SUBSYSTEM=="usb", ATTR{idVendor}=="0502", MODE="0664", GROUP="plugdev"
#ASUS
SUBSYSTEM=="usb", ATTR{idVendor}=="0b05", MODE="0664", GROUP="plugdev"
#Dell
SUBSYSTEM=="usb", ATTR{idVendor}=="413c", MODE="0664", GROUP="plugdev"
#Foxconn
SUBSYSTEM=="usb", ATTR{idVendor}=="0489", MODE="0664", GROUP="plugdev"
#Fujitsu & Fujitsu Toshiba
SUBSYSTEM=="usb", ATTR{idVendor}=="04c5", MODE="0664", GROUP="plugdev"
#Garmin-Asus
SUBSYSTEM=="usb", ATTR{idVendor}=="091e", MODE="0664", GROUP="plugdev"
#Google
SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", MODE="0664", GROUP="plugdev"
#Haier
SUBSYSTEM=="usb", ATTR{idVendor}=="201e", MODE="0664", GROUP="plugdev"
#Hisense
SUBSYSTEM=="usb", ATTR{idVendor}=="109b", MODE="0664", GROUP="plugdev"
#HTC
SUBSYSTEM=="usb", ATTR{idVendor}=="0bb4", MODE="0664", GROUP="plugdev"
#Huawei
SUBSYSTEM=="usb", ATTR{idVendor}=="12d1", MODE="0664", GROUP="plugdev"
#Intel
SUBSYSTEM=="usb", ATTR{idVendor}=="8087", MODE="0664", GROUP="plugdev"
#K-Touch
SUBSYSTEM=="usb", ATTR{idVendor}=="24e3", MODE="0664", GROUP="plugdev"
#KT Tech
SUBSYSTEM=="usb", ATTR{idVendor}=="2116", MODE="0664", GROUP="plugdev"
#Kyocera
SUBSYSTEM=="usb", ATTR{idVendor}=="0482", MODE="0664", GROUP="plugdev"
#Lenovo
SUBSYSTEM=="usb", ATTR{idVendor}=="17ef", MODE="0664", GROUP="plugdev"
#LG
SUBSYSTEM=="usb", ATTR{idVendor}=="1004", MODE="0664", GROUP="plugdev"
#Motorola
SUBSYSTEM=="usb", ATTR{idVendor}=="22b8", MODE="0664", GROUP="plugdev"
#MTK
SUBSYSTEM=="usb", ATTR{idVendor}=="0e8d", MODE="0664", GROUP="plugdev"
#NEC
SUBSYSTEM=="usb", ATTR{idVendor}=="0409", MODE="0664", GROUP="plugdev"
#Nook
SUBSYSTEM=="usb", ATTR{idVendor}=="2080", MODE="0664", GROUP="plugdev"
#Nvidia
SUBSYSTEM=="usb", ATTR{idVendor}=="0955", MODE="0664", GROUP="plugdev"
#OTGV
SUBSYSTEM=="usb", ATTR{idVendor}=="2257", MODE="0664", GROUP="plugdev"
#Pantech
SUBSYSTEM=="usb", ATTR{idVendor}=="10a9", MODE="0664", GROUP="plugdev"
#Pegatron
SUBSYSTEM=="usb", ATTR{idVendor}=="1d4d", MODE="0664", GROUP="plugdev"
#Philips
SUBSYSTEM=="usb", ATTR{idVendor}=="0471", MODE="0664", GROUP="plugdev"
#PMC-Sierra
SUBSYSTEM=="usb", ATTR{idVendor}=="04da", MODE="0664", GROUP="plugdev"
#Qualcomm
SUBSYSTEM=="usb", ATTR{idVendor}=="05c6", MODE="0664", GROUP="plugdev"
#SK Telesys
SUBSYSTEM=="usb", ATTR{idVendor}=="1f53", MODE="0664", GROUP="plugdev"
#Samsung
SUBSYSTEM=="usb", ATTR{idVendor}=="04e8", MODE="0664", GROUP="plugdev"
#Sharp
SUBSYSTEM=="usb", ATTR{idVendor}=="04dd", MODE="0664", GROUP="plugdev"
#Sony
SUBSYSTEM=="usb", ATTR{idVendor}=="054c", MODE="0664", GROUP="plugdev"
#Sony Ericsson
SUBSYSTEM=="usb", ATTR{idVendor}=="0fce", MODE="0664", GROUP="plugdev"
#Teleepoch
SUBSYSTEM=="usb", ATTR{idVendor}=="2340", MODE="0664", GROUP="plugdev"
#Toshiba
SUBSYSTEM=="usb", ATTR{idVendor}=="0930", MODE="0664", GROUP="plugdev"
#Wileyfox
SUBSYSTEM=="usb", ATTR{idVendor}=="2970", MODE="0664", GROUP="plugdev"
#YU
SUBSYSTEM=="usb", ATTR{idVendor}=="1ebf", MODE="0664", GROUP="plugdev"
#ZTE
SUBSYSTEM=="usb", ATTR{idVendor}=="19d2", MODE="0664", GROUP="plugdev"
#ZUK
SUBSYSTEM=="usb", ATTR{idVendor}=="2b4c", MODE="0664", GROUP="plugdev"

Troubleshooting

1. Make sure your device is connected and accessible via usb. lsusb should show a list of connected devices. The section after 'ID' in the output should match one of the idVendor numbers from the udev rules. For example, a Nexus One with idVendor 18d1 should return something like:

Bus 001 Device 005: ID 17ef:480d Lenovo Integrated Webcam [R5U877]
Bus 002 Device 002: ID 0bda:0119 Realtek Semiconductor Corp. 
Bus 002 Device 009: ID 18d1:4e12 Google Inc. Nexus One (debug)
Bus 004 Device 003: ID 0a5c:2145 Broadcom Corp. Bluetooth with Enhanced Data Rate II
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

2. If the device is detected in lsusb, try running adb devices. If you get ???????????? device, try reloading the udev rules:

 sudo sh -c "(udevadm control --reload-rules && udevadm trigger --action=change)"

You can also try disconnecting and reconnecting the usb cable to your device.

3. If you cannot access your device via adb, even after adding your linux user to the plugdev group and restarting the computer, you can try starting the adb service as root. This is dangerous and not recommended:

adb kill-server
sudo $(which adb) start-server
adb devices

Similarly, if fastboot devices returns no permissions, try running fastboot as root:

sudo $(which fastboot) devices