Archive

Archive for the ‘predictions’ Category

Why you shouldn’t be too pessimistic

May 13, 2021 2 comments

In our math research we make countless choices. We chose a problem to work on, decide whether its claim is true or false, what tools to use, what earlier papers to study which might prove useful, who to collaborate with, which computer experiments might be helpful, etc. Choices, choices, choices… Most our choices are private. Others are public. This blog is about wrong public choices that I made misjudging some conjectures by being overly pessimistic.

The meaning of conjectures

As I have written before, conjectures are crucial to the developments of mathematics and to my own work in particular. The concept itself is difficult, however. While traditionally conjectures are viewed as some sort of “unproven laws of nature“, that comparison is widely misleading as many conjectures are descriptive rather than quantitative. To understand this, note the stark contrast with experimental physics, as many mathematical conjectures are not particularly testable yet remain quite interesting. For example, if someone conjectures there are infinitely many Fermat primes, the only way to dissuade such person is to actually disprove the claim.

There is also an important social aspect of conjecture making. For a person who poses a conjecture, there is a certain clairvoyance respected by other people in the area. Predictions are never easy, especially of a precise technical nature, so some bravery or self-assuredness is required. Note that social capital is spent every time a conjecture is posed. In fact, a lot of it is lost when it’s refuted, you come out even if it’s proved relatively quickly, and you gain only if the conjecture becomes popular or proved possibly many years later. There is also a “boy who cried wolf” aspect for people who make too many conjectures of dubious quality — people will just tune out.

Now, for the person working on a conjecture, there is also a betting aspect one cannot ignore. As in, are you sure you are working in the right direction? Perhaps, the conjecture is simply false and you are wasting your time… I wrote about this all before in the post linked above, and the life/career implications on the solver are obvious. The success in solving a well known conjecture is often regarded much higher than a comparable result nobody asked about. This may seem unfair, and there is a bit of celebrity culture here. Thinks about it this way — two lead actors can have similar acting skills, but the one who is a star will usually attract a much larger audience…

Stories of conjectures

Not unlike what happens to papers and mathematical results, conjectures also have stories worth telling, even if these stories are rarely discussed at length. In fact, these “conjecture stories” fall into a few types. This is a little bit similar to the “types of scientific papers” meme, but more detailed. Let me list a few scenarios, from the least to the most mathematically helpful:

(1) Wishful thinking. Say, you are working on a major open problem. You realize that a famous conjecture A follows from a combination of three conjectures B, C and D whose sole motivation is their applications to A. Some of these smaller conjectures are beyond the existing technology in the area and cannot be checked computationally beyond a few special cases. You then declare that this to be your “program” and prove a small special case of C. Somebody points out that D is trivially false. You shrug, replace it with a weaker D’ which suffices for your program but is harder to disprove. Somebody writes a long state of the art paper disproving D’. You shrug again and suggest an even weaker conjecture D”. Everyone else shrugs and moves on.

(2) Reconfirming long held beliefs. You are working in a major field of study aiming to prove a famous open problem A. Over the years you proved a number of special cases of A and became one the leaders of the area. You are very optimistic about A discussing it in numerous talks and papers. Suddenly A is disproved in some esoteric situations, undermining the motivation of much of your older and ongoing work. So you propose a weaker conjecture A’ as a replacement for A in an effort to salvage both the field and your reputation. This makes happy everyone in the area and they completely ignore the disproof of A from this point on, pretending it’s completely irrelevant. Meanwhile, they replace A with A’ in all subsequent papers and beamer talk slides.

(3) Accidental discovery. In your ongoing work you stumble at a coincidence. It seem, all objects of a certain kind have some additional property making them “nice“. You are clueless why would that be true, since being nice belongs to another area X. Being nice is also too abstract to be checked easily on a computer. You consult a colleague working in X whether this is obvious/plausible/can be proved and receive No/Yes/Maybe answers to these three questions. You are either unable to prove the property or uninterested in problem, or don’t know much about X. So you mention it in the Final Remarks section of your latest paper in vain hope somebody reads it. For a few years, every time you meet somebody working in X you mention to them your “nice conjecture”, so much that people laugh at you behind your back.

(4) Strong computational evidence. You are doing computer experiments related to your work. Suddenly certain numbers appear to have an unexpectedly nice formula or a generating function. You check with OEIS and the sequence is there indeed, but not with the meaning you wanted. You use the “scientific method” to get a few more terms and they indeed support your conjectural formula. Convinced this is not an instance of the “strong law of small numbers“, you state the formula as a conjecture.

(5) Being contrarian. You think deeply about famous conjecture A. Not only your realize that there is no way one can approach A in full generality, but also that it contradicts some intuition you have about the area. However, A was stated by a very influential person N and many people believe in A proving it in a number of small special cases. You want to state a non-A conjecture, but realize the inevitable PR disaster of people directly comparing you to N. So you either state that you don’t believe in A, or that you believe in a conjecture B which is either slightly stronger or slightly weaker than non-A, hoping the history will prove you right.

(6) Being inspirational. You think deeply about the area and realize that there is a fundamental principle underlying certain structures in your work. Formalizing this principle requires a great deal of effort and results in a conjecture A. The conjecture leads to a large body of work by many people, even some counterexamples in esoteric situations, leading to various fixes such as A’. But at that point A’ is no longer the goal but more of a direction in which people work proving a number of A-related results.

Obviously, there are many other possible stories, while some stories are are a mixture of several of these.

Why do I care? Why now?

In the past few years I’ve been collecting references to my papers which solve or make some progress towards my conjectures and open problems, putting links to them on my research page. Turns out, over the years I made a lot of those. Even more surprisingly, there are quite a few papers which address them. Here is a small sampler, in random order:

(1) Scott Sheffield proved my ribbon tilings conjecture.

(2) Alex Lubotzky proved my conjecture on random generation of a finite group.

(3) Our generalized loop-erased random walk conjecture (joint with Igor Gorodezky) was recently proved by Heng Guo and Mark Jerrum.

(4) Our Young tableau bijections conjecture (joint with Ernesto Vallejo) was resolved by André Henriques and Joel Kamnitzer.

(5) My size Ramsey numbers conjecture led to a series of papers, and was completely resolved only recently by Nemanja Draganić, Michael Krivelevich and Rajko Nenadov.

(6) One of my partition bijection problems was resolved by Byungchan Kim.

The reason I started collecting these links is kind of interesting. I was very impressed with George Lusztig and Richard Stanley‘s lengthy writeups about their collected papers that I mentioned in this blog post. While I don’t mean to compare myself to these giants, I figured the casual reader might want to know if a conjecture in some paper had been resolved. Thus the links on my website. I recommend others also do this, as a navigational tool.

What gives?

Well, looks like none of my conjectures have been disproved yet. That’s a good news, I suppose. However, by going over my past research work I did discover that on three occasions when I was thinking about other people’s conjectures, I was much too negative. This is probably the result of my general inclination towards “negative thinking“, but each story is worth telling.

(i) Many years ago, I spent some time thinking about Babai’s conjecture which states that there are universal constants C, c >0, such that for every simple group G and a generating set S, the diameter of the Cayley graph Cay(G,S) is at most C(log |G|)c. There has been a great deal of work on this problem, see e.g. this paper by Sean Eberhard and Urban Jezernik which has an overview and references.

Now, I was thinking about the case of the symmetric group trying to apply arithmetic combinatorics ideas and going nowhere. In my frustration, in a talk I gave (Galway, 2009), I wrote on the slides that “there is much less hope” to resolve Babai’s conjecture for An than for simple groups of Lie type or bounded rank. Now, strictly speaking that judgement was correct, but much too gloomy. Soon after, Ákos Seress and Harald Helfgott proved a remarkable quasi-polynomial upper bound in this case. To my embarrassment, they referenced my slides as a validation of the importance of their work.

Of course, Babai’s conjecture is very far from being resolved for An. In fact, it is possible that the diameter is always O(n2). We just have no idea. For simple groups of Lie type or large rank the existing worst case diameter bounds are exponential and much too weak compared to the desired bound. As Eberhard and Jezernik amusingly wrote in the paper linked above, “we are still exponentially stupid“…

(ii) When he was my postdoc at UCLA, Alejandro Morales told me about a curious conjecture in this paper (Conjecture 5.1), which claimed that the number of certain nonsingular matrices over the finite field Fq is polynomial in q with positive coefficients. He and coauthors proved the conjecture is some special cases, but it was wide open in full generality.

Now, I thought about this type of problems before and was very skeptical. I spent a few days working on the problem to see if any of my tools can disprove it, and failed miserably. But in my stubbornness I remained negative and suggested to Alejandro that he should drop the problem, or at least stop trying to prove rather than disprove the conjecture. I was wrong to do that.

Luckily, Alejandro ignored my suggestion and soon after proved the polynomial part of the conjecture together with Joel Lewis. Their proof is quite elegant and uses certain recurrences coming from the rook theory. These recurrences also allow a fast computation of these polynomials. Consequently, the authors made a number of computer experiments and disproved the positivity of coefficients part of the conjecture. So the moral is not to be so negative. Sometimes you need to prove a positive result first before moving to the dark side.

(iii) The final story is about the beautiful Benjamini conjecture in probabilistic combinatorics. Roughly speaking, it says that for every finite vertex transitive graph G on n vertices and diameter O(n/log n) the critical percolation constant pc <1. More precisely, the conjecture claims that there is p<1-ε, such that a p-percolation on G has a connected component of size >n/2 with probability at least δ, where constants ε, δ>0 depend on the constant implied by the O(*) notation, but not on n. Here by “p-percolation” we mean a random subgraph of G with probability p of keeping and 1-p of deleting an edge, independently for all edges of G.

Now, Itai Benjamini is a fantastic conjecture maker of the best kind, whose conjectures are both insightful and well motivated. Despite the somewhat technical claim, this conjecture is quite remarkable as it suggested a finite version of the “pc<1″ phenomenon for infinite groups of superlinear growth. The latter is the famous Benjamini–Schramm conjecture (1996), which was recently proved in a remarkable breakthrough by Hugo Duminil-Copin, Subhajit Goswami, Aran Raoufi, Franco Severo and Ariel Yadin. While I always believed in that conjecture and even proved a tiny special case of it, finite versions tend to be much harder in my experience.

In any event, I thought a bit about the Benjamini conjecture and talked to Itai about it. He convinced me to work on it. Together with Chis Malon, we wrote a paper proving the claim for some Cayley graphs of abelian and some more general classes of groups. Despite our best efforts, we could not prove the conjecture even for Cayley graphs of abelian groups in full generality. Benjamini noted that the conjecture is tight for products of two cyclic groups, but that justification did not sit well with me. There seemed to be no obvious way to prove the conjecture even for the Cayley graph of Sn generated by a transposition and a long cycle, despite the very small O(n2) diameter. So we wrote in the introduction: “In this paper we present a number of positive results toward this unexpected, and, perhaps, overly optimistic conjecture.”

As it turns out, it was us who were being overly pessimistic, even if we never actually stated that we believe the conjecture is false. Most recently, in an amazing development, Tom Hutchcroft and Matthew Tointon proved a slightly weaker version of the conjecture by adapting the methods of Duminil-Copin et al. They assume the O(n/(log n)c) upper bound on the diameter which they prove is sufficient, for some universal constant c>1. They also extend our approach with Malon to prove the conjecture for all Cayley graphs of abelian groups. So while the Benjamini conjecture is not completely resolved, my objections to it are no longer valid.

Final words on this

All in all, it looks like I was never formally wrong even if I was a little dour occasionally (Yay!?). Turns out, some conjectures are actually true or at least likely to hold. While I continue to maintain that not enough effort is spent on trying to disprove the conjectures, it is very exciting when they are proved. Congratulations to Harald, Alejandro, Joel, Tom and Matthew, and posthumous congratulations to Ákos for their terrific achievements!

It could have been worse! Academic lessons of 2020

December 20, 2020 4 comments

Well, this year sure was interesting, and not in a good way. Back in 2015, I wrote a blog post discussing how video talks are here to stay, and how we should all agree to start giving them and embrace watching them, whether we like it or not. I was right about that, I suppose. OTOH, I sort of envisioned a gradual acceptance of this practice, not the shock therapy of a phase transition. So, what happened? It’s time to summarize the lessons and roll out some new predictions.

Note: this post is about the academic life which is undergoing some changes. The changes in real life are much more profound, but are well discussed elsewhere.

Teaching

This was probably the bleakest part of the academic life, much commented upon by the media. Good thing there is more to academia than teaching, no matter what the ignorant critics think. I personally haven’t heard anyone saying post-March 2020, that online education is an improvement. If you are like me, you probably spent much more time preparing and delivering your lectures. The quality probably suffered a little. The students probably didn’t learn as much. Neither party probably enjoyed the experience too much. They also probably cheated quite a bit more. Oh, well…

Let’s count the silver linings. First, it will all be over some time next year. At UCLA, not before the end of Summer. Maybe in the Fall… Second, it could’ve been worse. Much worse. Depending on the year, we would have different issues. Back in 1990, we would all be furloughed for a year living off our savings. In 2000, most families had just one personal computer (and no smartphones, obviously). Let the implications of that sink in. But even in 2010 we would have had giant technical issues teaching on Skype (right?) by pointing our laptop cameras on blackboards with dismal effect. The infrastructure which allows good quality streaming was also not widespread (people were still using Redbox, remember?)

Third, the online technology somewhat mitigated the total disaster of studying in the pandemic time. Students who are stuck in faraway countries or busy with family life can watch stored videos of lectures at their convenience. Educational and grading software allows students to submit homeworks and exams online, and instructors to grade them. Many other small things not worth listing, but worth being thankful for.

Fourth, the accelerated embrace of the educational technology could be a good thing long term, even when things go back to normal. No more emails with scanned late homeworks, no more canceled/moved office hours while away at conferences. This can all help us become better at teaching.

Finally, a long declared “death of MOOCs” is no longer controversial. As a long time (closeted) opponent to online education, I am overjoyed that MOOCs are no longer viewed as a positive experience for university students, more like something to suffer through. Here in CA we learned this awhile ago, as the eagerness of the current Gov. Newsom (back then Lt. Gov.) to embrace online courses did not work out well at all. Back in 2013, he said that the whole UC system needs to embrace online education, pronto: “If this doesn’t wake up the U.C. [..] I don’t know what will.” Well, now you know, Governor! I guess, in 2020, I don’t have to hide my feelings on this anymore…

Research

I always thought that mathematicians can work from anywhere with a good WiFi connection. True, but not really – this year was a mixed experience as lonely introverts largely prospered research wise, while busy family people and extraverts clearly suffered. Some day we will know how much has research suffered in 2020, but for me personally it wasn’t bad at all (see e.g. some of my results described in my previous blog post).

Seminars

I am not even sure we should be using the same word to describe research seminars during the pandemic, as the experience of giving and watching math lectures online are so drastically different compared to what we are used to. Let’s count the differences, which are both positive and negative.

  1. The personal interactions suffer. Online people are much more shy to interrupt, follow up with questions after the talk, etc. The usual pre- or post-seminar meals allow the speaker to meet the (often junior) colleagues who might be more open to ask questions in an informal setting. This is all bad.
  2. Being online, the seminar opened to a worldwide audience. This is just terrific as people from remote locations across the globe now have the same access to seminars at leading universities. What arXiv did to math papers, covid did to math seminars.
  3. Again, being online, the seminars are no longer restricting themselves to local speaks or having to make travel arrangements to out of town speakers. Some UCLA seminars this year had many European speakers, something which would be prohibitively expensive just last year.
  4. Many seminars are now recorded with videos and slides posted online, like we do at the UCLA Combinatorics and LA Combinatorics and Complexity seminars I am co-organizing. The viewers can watch them later, can fast forward, come back and re-watch them, etc. All the good features of watching videos I extolled back in 2015. This is all good.
  5. On a minor negative side, the audience is no longer stable as it varies from seminar to seminar, further diminishing personal interactions and making level of the audience somewhat unpredictable and hard to aim for.
  6. As a seminar organizer, I make it a personal quest to encourage people to turn on their cameras at the seminars by saying hello only to those whose faces I see. When the speaker doesn’t see the faces, whether they are nodding or quizzing, they are clueless whether the they are being clear, being too fast or too slow, etc. Stopping to ask for questions no longer works well, especially if the seminar is being recorded. This invariably leads to worse presentations as the speakers can misjudge the audience reactions.
  7. Unfortunately, not everyone is capable of handling technology challenges equally well. I have seen remarkably well presented talks, as well as some of extremely poor quality talks. The ability to mute yourself and hide behind your avatar is the only saving grace in such cases.
  8. Even the true haters of online educations are now at least semi-on-board. Back in May, I wrote to Chris Schaberg dubbed by the insufferable Rebecca Schuman as “vehemently opposed to the practice“. He replied that he is no longer that opposed to teaching online, and that he is now in a “it’s really complicated!” camp. Small miracles…

Conferences

The changes in conferences are largely positive. Unfortunately, some conferences from the Spring and Summer of 2020 were canceled and moved, somewhat optimistically, to 2021. Looking back, they should all have been held in the online format, which opens them to participants from around the world. Let’s count upsides and downsides:

  1. No need for travel, long time commitments and financial expenses. Some conferences continue charging fees for online participation. This seems weird to me. I realize that some conferences are vehicles to support various research centers and societies. Whatever, this is unsustainable as online conferences will likely survive the pandemic. These organizations should figure out some other income sources or die.
  2. The conferences are now truly global, so the emphasis is purely on mathematical areas than on the geographic proximity. This suggests that the (until recently) very popular AMS meetings should probably die, making AMS even more of a publisher than it is now. I am especially looking forward to the death of “joint meetings” in January which in my opinion outlived their usefulness as some kind of math extravaganza events bringing everyone together. In fact, Zoom simply can’t bring five thousand people together, just forget about it…
  3. The conferences are now open to people in other areas. This might seem minor — they were always open. However, given the time/money constraints, a mathematician is likely to go only to conferences in their area. Besides, since they rarely get invited to speak at conferences in other areas, travel to such conferences is even harder to justify. This often leads to groupthink as the same people meet year after year at conferences on narrow subjects. Now that this is no longer an obstacle, we might see more interactions between the fields.
  4. On a negative side, the best kind of conferences are small informal workshops (think of Oberwolfach, AIM, Banff, etc.), where the lectures are advanced and the interactions are intense. I miss those and hope they come back as they are really irreplaceable in the only setting. If all goes well, these are the only conferences which should definitely survive and even expand in numbers perhaps.

Books and journals

A short summary is that in math, everything should be electronic, instantly downloadable and completely free. Cut off from libraries, thousands of mathematicians were instantly left to the perils of their university library’s electronic subscriptions and their personal book collections. Some fared better than others, in part thanks to the arXiv, non-free journals offering old issues free to download, and some ethically dubious foreign websites.

I have been writing about my copyleft views for a long time (see here, there and most recently there). It gets more and more depressing every time. Just when you think there is some hope, the resilience of paid publishing and reluctance to change by the community is keeping the unfortunate status quo. You would think everyone would be screaming about the lack of access to books/journals, but I guess everyone is busy doing something else. Still, there are some lessons worth noting.

  1. You really must have all your papers freely available online. Yes, copyrighted or not, the publishers are ok with authors posting their papers on their personal website. They are not ok when others are posting your papers on their websites, so the free access to your papers is on you and your coauthors (if any). Unless you have already done so, do this asap! Yes, this applies even to papers accessible online by subscription to selected libraries. For example, many libraries including all of UC system no longer have access to Elsevier journals. Please help both us and yourself! How hard is it to put the paper on the arXiv or your personal website? If people like Noga Alon and Richard Stanley found time to put hundreds of their papers online, so can you. I make a point of emailing to people asking them to do that every time I come across a reference which I cannot access. They rarely do, and usually just email me the paper. Oh, well, at least I tried…
  2. Learn to use databases like MathSciNet and Zentralblatt. Maintain your own website by adding the slides, video links as well as all your papers. Make sure to clean up and keep up to date your Google Scholar profile. When left unattended it can get overrun with random papers by other people, random non-research files you authored, separate items for same paper, etc. Deal with all that – it’s easy and takes just a few minutes (also, some people judge them). When people are struggling trying to do research from home, every bit of help counts.
  3. If you are signing a book contract, be nice to online readers. Make sure you keep the right to display a public copy on your website. We all owe a great deal of gratitude to authors who did this. Here is my favorite, now supplemented with high quality free online lectures. Be like that! Don’t be like one author (who will remain unnamed) who refused to email me a copy of a short 5 page section from his recent book. I wanted to teach the section in my graduate class on posets this Fall. Instead, the author suggested I buy a paper copy. His loss — I ended up teaching some other material instead. Later on, I discovered that the book is already available on one of those ethically compromised websites. He was fighting a battle he already lost!

Home computing

Different people can take different conclusions from 2020, but I don’t think anyone would argue the importance of having good home computing. There is a refreshing variety of ways in which people do this, and it’s unclear to me what is the optimal set up. With a vaccine on the horizon, people might be reluctant to further invest into new computing equipment (or video cameras, lights, whiteboard, etc.), but the holiday break is actually a good time to marinate on what worked out well and what didn’t.

Read your evaluations and take them to heart. Make changes when you see there are problems. I know, it’s unfair, your department might never compensate you for all this stuff. Still, it’s a small price to pay for having a safe academic job in the time of widespread anxiety.

Predictions for the future

  1. Very briefly: I think online seminars and conferences are here to stay. Local seminars and small workshops will also survive. The enormous AMS meetings and expensive Theory CS meetings will play with the format, but eventually turn online for good or die untimely death.
  2. Online teaching will remain being offered by every undergraduate math program to reach out to students across the spectrum of personal circumstances. A small minority of courses, but still. Maybe one section of each calculus, linear algebra, intro probability, discrete math, etc. Some faculty might actually prefer this format to stay away from office one semester. Perhaps, in place of a sabbatical, they can ask for permission to spend a semester some other campus, maybe in another state or country, while they continue teaching, holding seminars, supervising students, etc. This could be a perk of academic life to compete with the “remote work” that many businesses are starting to offer on a permanent basis. Universities would have to redefine what they mean by “residence” requirement for both faculty and students.
  3. More university libraries will play hardball and unsubscribe from major for-profit publishers. This would again sound hopeful, but not gain a snowball effect for at least the next 10 years.
  4. There will be some standardization of online teaching requirements across the country. Online cheating will remain widespread. Courts will repeatedly rule that business and institutions can discount or completely ignore all 2020 grades as unreliable in large part because of the cheating scandals.

Final recommendations

  1. Be nice to your junior colleagues. In the winner-take-all no-limits online era, the established and well-known mathematicians get invited over and over, while their junior colleagues get overlooked, just in time when they really need help (job market might be tough this year). So please go out of your way to invite them to give talks at your seminars. Help them with papers and application materials. At least reply to their emails! Yes, even small things count…
  2. Do more organizing if you are in position to do so. In the absence of physical contact, many people are too shy and shell-shocked to reach out. Seminars, conferences, workshops, etc. make academic life seem somewhat normal and the breaks definitely allow for more interactions. Given the apparent abundance of online events one my be forgiven to think that no more is needed. But more locally focused online events are actually important to help your communities. These can prove critical until everything is back to normal.

Good luck everybody! Hope 2021 will be better for us all!