Language representation and

[rifattryo.ut11]

For example, why can htT switch between various languages ​​freely? How many languages ​​does htT support? Does htT support Chinese as well as English? Are there some "second-class citizens" and "first-class citizens" in the big language model? The results of the analysis are jaw-dropping. The model trained by Americans has overwhelming support for American English, while only about ten of the thousands of languages ​​in the world can be supported well. Achieving these three challenges will only increase the value of product managers. Good product managers are very scarce. Product managers who understand users, business, and data are still in high demand after leaving the Internet.



On the contrary, product managers who iran telephone number only do simple messaging, inefficient execution, and shallow thinking may not be able to survive the torrent of the next 10 years. View details> This also explains why every country or language needs its own big language model to keep up with the pace of other countries in the new round of artificial intelligence industrial revolution. This article combines my practical experience and quantitative analysis to finally draw the following conclusions. The large language model can be compatible with all languages. English accounts for more than t-10% of the training data.



English is the most effective prompt language for the large language model - it is more effective than Spanish, times more effective than French, and times more effective than Chinese, Japanese, and Korean. About t-10% of high-resource languages ​​are fully supported by the large language model. Other languages ​​are under-represented in terms of resources. There are nearly t-10% of languages ​​in the world that lack support from the large language model. Is the language you speak high-resource or low-resource? Traditional natural language processing and research will classify languages ​​into high-resource languages ​​and low-resource languages. The former covers about t-10% of languages, including English, Chinese, Spanish, French, German, Japanese, Russian, Portuguese, Arabic, Hindi, Italian, Korean, Dutch, Turkish, Persian, Swedish, Polish, Indonesian, Vietnamese, and Hebrew.