Although few people paid attention when they organized the first Ultimate Fighting Championship, that changed quickly. The UFC offered a no-holds-barred form of competition, where under a minimal set of rules anyone could step in the octagon to match his skills against an opponent of any fighting style. Among the competitors were practitioners of Karate, Box, Judo, Kick-box, Savate, Wrestling, and other mainstream or niche martial arts. Adherents of every single one of them around the world, whose attachment to their chosen art often borders on religious, expected their style to emerge victorious. But the fighter who walked through his opponents on his way to the final victory was a physically unassuming Brazilian with a very strange fighting style. Every single match of him went down the same way. He avoided the punches with mixed levels of success until he managed to grab his opponent, dragged him to the ground, then quickly submitted him with a choke or some arm- or leglocks. The novelty wasn't in the techniques, which had already been part of Japanese Judo, Jiu-jitsu, and similar martial arts, but the approach to combat that put ground fighting at the center, instead of treating it as an auxiliary part of one's wider repertoire. The apparent lack of which made Royce Gracie's success all the more spectacular. He dominated the next 4 UFC events, proving that his first victory wasn't just a fluke.
Brazilian Jiu-Jitsu became a phenomenon almost overnight. Today, UFC is one of the most-viewed and most widely-known martial-arts events. BJJ has become ubiquitous, not only in the octagon, but also in movies, among celebrities, and most importantly among the dojo-s all around the world. But have the Gracies achieved their goal of proving that BJJ is the best fighting style of all? The answer is yes and no. UFC (and MMA-events in general), as good a controlled experiment in the field as imaginable, demonstrated repeatedly that an expert grappler will most probably beat a fighter of any other style if the latter has zero experience in ground fighting - for the reasons I'll try to inspect later. But the days are long gone when someone being very good only in grappling alone can get to the top. Royce Gracie of the 90s teleported to the present wouldn't last a minute against the world-class fighters of today. Everyone recognized that some form of grappling (BJJ, Sambo, Judo, ...) is an indispensable part of a fighter's repertoire at least as much, or even more than boxing-skills. Today's UFC champions are all expert grapplers, without exception.
Where had BJJ been before?
How did this happen? Humans have practiced and developed martial arts around the globe since at least ancient times. The idea that something completely new can still come up seemed naive, almost foolish. The answer to this question I think is twofold. First of all, as mentioned above, the core of BJJ's technical repertoire wasn't new at all. What new was is the sole emphasis on ground fighting. And - here comes some pure historical speculation - that is something that that in former times had less chance to appear. Ground fighting is only practical in one-on-one situations. During a raid, in battle, even on the street where attackers could jump on one at night, one can expect to face multiple opponents. Being entangled with one attacker on the ground leaves one completely exposed to the second one. Relying only on ground fighting skills pays off only in a ring. Prize-fighting has been around for millennia, too, but only modern times are peaceful enough to let people decouple sport fighting from survival skills. This does not mean to belittle the efficacy of BBJ. In a one-on-one situation, that arose even in older times frequently enough, it's proven to be superb.
Efficacy
And now, let's go into some speculations about why it is superior to stand-up styles. I think it has multiple, related reasons. These are the following: the asymmetry between grappling and kicking-punching styles, the way it's practiced, and - surprisingly enough - the more limited technical repertoire.
By asymmetry, I mean that stand-up fighting is more vulnerable to grappling techniques than the other way around. If a grappler can close the distance and drag his opponent to the floor, the stand-up fighter has very little chance to disentangle - that would require skills matching his attacker. Pull down a world-class boxer or karateka to the ground, and his hard-earned skills are rendered useless. It has been proved countless times in the octagon with boxing-legends like James Tooney, who lasted maybe a minute until choked unconscious by Randy Couture. Stand-up fighters often harbor the illusion that they could knock out any wrestler or grappler who got in punching range before he could grab them. This belief again has been proven to be false. Actually, it's very hard to knock someone out. The grappler may suffer one or two blows to the head before he manages to lay his hands on his opponent, but they are very rarely enough to end the fight, and then the striker has no time to land another punch.
The other part of this asymmetry is that humans seem to be much more in their element with punching than with grappling. Anyone can swing his arm and break a nose with a little luck. My former boxing coach said once that a complete beginner can give him a black eye if he drops his guard at the wrong time. But put someone on the mat for the first time, and he won't have a clue what to do if he's not supposed to use his fists. Human instincts in grappling don't go further than brute-force strangling. Give someone 6 months of boxing, and he will win against a complete beginner (all other things being equal) most of the times - but not necessarily all the time, especially if the opponent is more aggressive and mentally tougher. But train someone in BJJ for the same duration and he will win 10 times out of 10 against someone with the same physical abilities - aggressivity notwithstanding.
The second advantage of BJJ is the training-style. According to my experience, half of any training session is free fighting with constantly changing partners. This puts very hard selection pressure on both the fighters and the techniques. No technique that is ineffective gets to the mainstream, regardless of how "elegant" or effective it looks. This is in very stark contrast to how traditional martial arts are taught, where a frequent answer to a student's question inquiring about the technique is: "because it has been taught like this for a hundred years". Techniques don't get tested immediately against opponents who do their best to counter them. Consequently, the world of traditional eastern martial arts is full of cringe-worthy black-belts. Such "masters" don't exist among BJJ-ers, or not for long at least. No one will want to put on a brown-belt if his ass gets kicked by blue-belts at every single class. The "am I a fraud?"-complex that plagues many karatekas ("can I actually defend myself outside of the dojo?") is completely unknown in the BJJ world (or as a matter of fact, in the world of boxing, wrestling, thai-box, or any kind of art that focuses on actual fighting instead of forms and katas).
Finally, I made a surprising statement above about the positive effect of the more-constrained toolset. It manifests in multiple ways. First of all, because kicking and striking are not part of the repertoire, you can spar at 90% every day without serious risk of injuring your partner or getting injured yourself. In boxing, the fighters either hold back during sparring sessions or spend more time recovering from concussions or fractured ribs than in the ring. The second reason requires a bit longer explanation. In stand-up fighting styles, a big part of the defense techniques are developed against the style's own attacks - and vice-versa. A sparring session between advanced Wing-Chun or Kali practitioners looks very impressive. The opponents stand at close quarters and exchange blows and parries with such speed that spectators can only see a blur. But those parries were developed for exactly that types of strikes and they are much less effective against, let's say, a boxer's hook or upper-cut or a thai-boxer's roundhouse kick. Put an experienced karateka, a boxer, a Wing-Chun student against someone whose rhythm, style, simply the way he moves his body are radically different from the one he got used to, and you will see instant confusion and drop in confidence. This phenomenon might exist in the grappling world, but to a much lesser extent. To put it simply, there are only so many ways to twist an arm or place a choke - the "beauty" of a move or the adherence of it to some abstract principle - in short: style - are just meaningless there.
Popularity
The efficacy alone is not enough to explain BJJ's popularity. There is also a "fun"-factor in it. It is a cerebral sport. Techniques have counter-techniques that in turn have their own counters, and so on. Aggressivity plays a smaller role than to most martial arts. Few people enjoy boxing-sparring, especially at first, but almost everyone I have seen trying it did enjoy rolling (sparring in BJJ) - especially at first. There is something in it that makes you feel placed back in your childhood. For reasons elaborated above, it also has a very quick learning curve. Usually, the moment a "master" starts explaining to you that his art makes strength and weight difference irrelevant, is the moment you should politely turn on your heels and leave the dojo. But because of the natural human inaptitude in grappling, a strong 100kg man with no prior-experience could be completely at the mercy of a semi-advanced 70kg BJJ-practitioner. This is of course changes quickly, and between two experienced fighters weight matters as much as it does in boxing or wrestling.
The practical nature of BJJ is also more appealing to modern people. The relationship between master and student is much more informal than the rigidly hierarchical ones found as the default among eastern traditions. There are no formal exams - the master sees you perform every day and will grant you the belt when he thinks you deserve it. There are no katas, no endless repetitions of basic techniques against imaginary opponents, no infusion of a particular philosophy. But there still is a sense of belonging, the atmosphere of a dojo, and a natural hierarchy built on performance demonstrated every day.
Criticism
You can never satisfy everyone... Despite its huge success, BJJ, or rather the competition rules of BJJ competitions, are under heavy criticism nowadays - even from the Gracie-family. Many argue that it has become too "domesticated", too sportlike. Like Judo or Karate, it has lost its edge as a tool for self-defense. As usual, there is some truth in the criticism. Many dojos concentrate on ground-fighting to the extent that they don't even practice take-downs - as they don't score high in a competition. And many techniques in BBJ work only with the assumption that the opponent is prohibited from using his fists.
But the flipside of the coin is that these restrictions make the practice safe. As explained above, you can have a go at your partner at almost full throttle. Great boxers are not great because they know some "lethal moves" no one else does, but because their strength, speed, timing, sense of rhythm, and mastery of basic techniques surpass that of others. The same applies to any other martial art. People who are overly impressed by the brutality of certain styles (like Krav Maga), would better accept the fact that some of the most effective ways to incapacitate opponents will be never mastered by anyone. You will run out of training partners long before you practiced the art of eye-gauging to perfection.
Final worlds
I went through all the reasons why I find BJJ marvelous at length, but I think reiterating the best ones is not a bad way to wrap it up. It's a no-nonsense martial art. It's one of the most effective forms of self-defense, and it gives usable skills from day one. It hits the sweet spot between efficiency and safe practicability. It improves strength, agility, flexibility, endurance, and concentration skills. It keeps you on your toes and mentally fresh every day. But it is also something you can practice even in your senior years. And finally, and most importantly, rolling with your partner on the ground like a kid while trying to outsmart him at every move is pure, raw fun.