Allen was drafted in the 3rd round, 76th overall, by the San Diego Chargers in the 2013 NFL Draft. He was the highest selected wide receiver from Cal since DeSean Jackson in 2008. On May 9, 2013, he signed a four-year contract with the Chargers.
In Week 4 of 2014, Allen established a career high of 10 receptions for 135 yards in a 33–14 win over the Jacksonville Jaguars. During week 12 against the Ravens Allen had caught 2 touchdowns and 121 receiving yards contributing to the Chargers win. He missed the last two games of the season after suffering a broken right collarbone against Denver in Week 15. He finished the season with 77 receptions for 783 yards and 4 touchdowns.