人工智能的自我意识何以可能?

How Could AI Develop Its Self-consciousness?

人工智能危险之处不是能力,而是自我意识。只要人工智能拥有对自身系统的反思能力,就有可能改造自身系统,创造新规则,尤其是,如果人工智能发明一种属于自己的万能语言,能力相当于人类的自然语言,那么,所有的程序系统都可以通过它自己的万能语言加以重新理解、重新构造和重新定义,那么就非常危险。如果人工智能拥有了拟人化的情感、欲望和价值观,必然更为危险,因为人的欲望和价值观正是一切冲突的根源。

The danger of AI will come from its self-consciousness rather than its capacity.AI would be a challenge to mankind if it develops a capacity for reflection on the system of its own,so that it would know how to remake its system with new rules,especially when it could invent its own language for all purposes,equal in capacity to human natural language,it could do anything it would.And AI would be most dangerous if it would be able to learn human desires,emotions and values,because all hostilities are based upon human desires and values.