The club also removed the interim tag from general manager Tom Fitzgerald.
Ruff has worked as an assistant for the New York Rangers since 2017. He takes over for Alain Nasreddine, who was named interim head coach when John Hynes was fired in December.
The veteran head coach manned the bench for nearly 1,500 games between the Buffalo Sabres and Dallas Stars. The 60-year-old is sixth on the all-time wins list with 736 and took Buffalo to the Stanley Cup Final in 1999.
It was reported in June the Devils narrowed down their coaching search to four candidates, with Gerard Gallant, Peter Laviolette, and John Stevens known to have interviewed for the vacancy.
Meanwhile, Fitzgerald was promoted to interim GM in January after Ray Shero was fired. The organization conducted a search for a permanent GM, which reportedly included external candidates such as former GM of the Year Mike Gillis, but ultimately decided to keep Fitzgerald in charge.
Ruff and Fitzgerald know each other from their time with the Panthers. They spent four seasons together in Florida from 1993-97, when Fitzgerald was a player and Ruff was an assistant coach.
The Devils entered the 2019-20 campaign with sky-high expectations after landing Jack Hughes with the No. 1 pick and acquiring P.K. Subban in a blockbuster trade. However, New Jersey was unable to capitalize and finished the season in 26th place.