我正在处理屏幕抓取，并希望检索特定页面的源代码。
如何使用 javascript 实现这一目标？请帮我。
Javascript 可以读取任何网页的源代码吗？
IT技术
javascript
html
2021-01-26 16:27:48
6个回答
简单的入门方法，试试jQuery
$("#links").load("/Main_Page #jq-p-Getting-Started li");
更多在jQuery 文档
另一种以更加结构化的方式进行屏幕抓取的方法是使用YQL 或 Yahoo Query Language。它将返回结构化为 JSON 或 xml 的抓取数据。
例如，
让我们抓取 stackoverflow.com
select * from html where url="http://stackoverflow.com"
会给你一个像这样的 JSON 数组（我选择了那个选项）
"results": {
"body": {
"noscript": [
{
"div": {
"id": "noscript-padding"
}
},
{
"div": {
"id": "noscript-warning",
"p": "Stack Overflow works best with JavaScript enabled"
}
}
],
"div": [
{
"id": "notify-container"
},
{
"div": [
{
"id": "header",
"div": [
{
"id": "hlogo",
"a": {
"href": "/",
"img": {
"alt": "logo homepage",
"height": "70",
"src": "http://i.stackoverflow.com/Content/Img/stackoverflow-logo-250.png",
"width": "250"
}
……..
这样做的好处在于，您可以进行投影和 where子句，最终让您获得结构化的抓取数据，并且只获得您需要的数据（最终线路上的带宽要少得多），
例如
select * from html where url="http://stackoverflow.com" and
xpath='//div/h3/a'
会得到你
"results": {
"a": [
{
"href": "/questions/414690/iphone-simulator-port-for-windows-closed",
"title": "Duplicate: Is any Windows simulator available to test iPhone application? as a hobbyist who cannot afford a mac, i set up a toolchain kit locally on cygwin to compile objecti … ",
"content": "iphone\n simulator port for windows [closed]"
},
{
"href": "/questions/680867/how-to-redirect-the-web-page-in-flex-application",
"title": "I have a button control ....i need another web page to be redirected while clicking that button .... how to do that ? Thanks ",
"content": "How\n to redirect the web page in flex application ?"
},
…..
现在只得到我们做的问题
select title from html where url="http://stackoverflow.com" and
xpath='//div/h3/a'
注意投影中的标题
"results": {
"a": [
{
"title": "I don't want the function to be entered simultaneously by multiple threads, neither do I want it to be entered again when it has not returned yet. Is there any approach to achieve … "
},
{
"title": "I'm certain I'm doing something really obviously stupid, but I've been trying to figure it out for a few hours now and nothing is jumping out at me. I'm using a ModelForm so I can … "
},
{
"title": "when i am going through my project in IE only its showing errors A runtime error has occurred Do you wish to debug? Line 768 Error:Expected')' Is this is regarding any script er … "
},
{
"title": "I have a java batch file consisting of 4 execution steps written for analyzing any Java application. In one of the steps, I'm adding few libs in classpath that are needed for my co … "
},
{
……
编写查询后，它会为您生成一个 url
在我们的情况下。
所以最终你最终会做这样的事情
var titleList = $.getJSON(theAboveUrl);
和它一起玩。
漂亮，不是吗？
可以使用 Javascript，只要您通过域上的代理获取所需的任何页面：
<html>
<head>
<script src="/js/jquery-1.3.2.js"></script>
</head>
<body>
<script>
$.get("www.mydomain.com/?url=www.google.com", function(response) {
alert(response)
});
</script>
</body>
您可以简单地使用XmlHttp(AJAX) 来访问所需的 URL，来自 URL 的 HTML 响应将在responseText属性中可用。如果它不是同一个域，您的用户将收到一个浏览器警报，内容类似于“此页面正在尝试访问不同的域。您要允许吗？”
您可以使用fetch：
const URL = 'https://www.sap.com/belgique/index.html';
fetch(URL)
.then(res => res.text())
.then(text => {
console.log(text);
})
.catch(err => console.log(err));作为安全措施，Javascript 无法读取来自不同域的文件。虽然可能有一些奇怪的解决方法，但我会考虑为这项任务使用不同的语言。
其它你可能感兴趣的问题