[See Part One: Things Java Got Right for the beginning of this piece]
I think the Swing experiment has been a failure. I know that many people disagree, but probably every Mac user agrees. Cross-platform user interfaces do not seem to be possible. I agree with Raph Levien: provide a way to keep the bulk of the code shared and still write a native interface for each platform. Skype has done this really well. They have a client program that works on Linux, Windows, and Mac OS X, and is native on each of them. As a result, Skype on Mac OS X is a pleasure to use. SWT is a better approach than Swing, for this reason. However, I think it still tries to share too much between platforms.
Sun marketed Java as a language. Microsoft made a better decision: they are marketing the CLR as a platform. This is an excellent idea, but I do not believe it is perfect. The problem is that the CLR and JVM are quite restrictive about how code executes and how data can be manipulated. This means that while they can support multiple languages, they all end up working like C# or Java. This isn't completely true, as IronPython or Jython are fairly different than their runtimes, but there are some interoperability issues. For example, using a Jython class from Python is not as easy as you might hope. Some amount of glue code will always be required when calling between languages, but it would be nice to reduce it to a minimum. Tim Bray has more comments on this issue.
Personally, I think that a low-level approach, like the Low-Level Virtual Machine (LLVM), would make a better platform than the JVM. The reason is that LLVM is a closer match to the underlying hardware. This enables it to execute C or C++ programs, or to be used as a CPU instruction set. The JVM, on the other hand, can only represent things that can be shoehorned onto the Java model. I think using LLVM or something similar would lead to better interoperability with native code, and enable new languages to be implemented more efficiently.
Originally, Sun really pushed Java's "write once, run anywhere" motto. As part of this strategy, they actively discouraged developers from integrating Java with existing code. Today, the way to interface with native code is via the Java Native Interface (JNI), which is not exactly easy to use. It would have been very advantageous to provide some easier mechanism. However, that would have been counter to Sun's cross-platform goals. This was more or less the heart of the Sun/Microsoft Java lawsuit which took years to be resolved at the cost of $1.6 billion USD for Microsoft. As James Gosling notes, it is an advantage that Java programs are not tied to one operating system or CPU. However, as a developer, I sympathize with Microsoft's original position: Java is a good language, so why can't I easily integrate it with my existing projects?
This is another part of Sun's portability game. According to them, Java programs must only be distributed as bytecode, which leads to two problems. First, it means that users must download the JRE in order to use Java software. Even though downloads are much faster today than in 1995, it still makes it much harder for a user to download a new Java application than it is to grab BitTorrent. It would be much nicer if developers could optionally build a single native executable. Even if that executable used Python's trick, and distributed the bare minimum components of the JRE, it be wonderful.
The second reason that JIT-Only is bad is that it makes sharing code in memory more difficult. Native code that is compiled into shared libraries only gets loaded into memory once, no matter how many programs use them. Additionally, it only needs to be compiled once. Since Java code must be compiled by the JIT, each process has its own copy. I think Microsoft's on-disk code cache approach with the CLR is a winner here.
Interestingly, everything on this list is an implementation issue, not problems with the language itself. This means that it would be possible for Sun, or someone else, to fix them. For example, GCJ allows Java to be compiled to native code. However, I do not see Sun or any other major organization supporting any major changes to Java in the near future. I hope to be proven wrong.
At this point, Java is well entrenched and widely used. This means that it will need to live on forever, even if it does end up being abandoned. I think Java's main strength is that it is an excellent "medium-level" language. It is ideal for software that needs good performance, but not the best possible performance. Since it has high-level features, the code will have less bugs and programmers will be more productive, but not as productive as they would be in a high-level language. Since it is statically typed, teh program will likely run faster than it would if written in a high-level language, but not as fast as tuned C code. There is a significant amount of software that fits into this category, and Java is ideal for it. However, I think there is far, far more software whose performance basically does not matter. For these applications, an even higher level language would be a much better choice.
At the very low end of the spectrum, where interfacing with the hardware and the best possible performance is required, C is still the best thing for the job. However, advances in technology mean that even there, C is continuing to lose ground. For example, the Jikes RVM is a virtual machine written entirely in Java, and Singularity is an operating system written in C#. They both show performance that is comparable to that of systems written in C. So it is possible that Java, or something like it, could eventually replace C. The biggest challenge is that there is an enormous amount of software that needs to continue to run and to interface with new code. This is the issue that will prevent Java from growing "down" and displacing C: it doesn't play well with native code.