您的当前位置:首页>全部文章>文章详情

ThinkPHP6+JS实现大文件分片上传,切片上传

发表于:2023-02-23 23:00:59浏览:1413次TAG: #ThinkPHP #layui #文件上传 #大文件上传 #切片上传

在日常的项目中,对于很大的文件比如几百M的音频、视频、软件上传,如果直接上传到服务器,经常处理不了。可以在客户端先把大文件切割成小文件一个一个上传,然后服务器端再组合成一个大文件,同时支持上传取消删除已上传的临时文件。

图片alt

<div class="layui-input-block">
    <button type="button" class="layui-btn" id="fileUpload">上传文件</button>
</div>
$('#fileUpload').on('click',function(){
    uploadChunk();        
})

    function uploadChunk(){
            layer.open({
                'title':'上传文件',
                'type':1,
                'area': ['600px', '320px'],
                'content':`<div class="layui-form p-3">
                        <div class="layui-form-item">
                            <label class="layui-form-label">来源:</label>
                            <div class="layui-input-block">
                                <input type="radio" name="uploadtype" lay-filter="type" value="1" title="本地上传" checked>
                                <input type="radio" name="uploadtype" lay-filter="type" value="2" title="网络文件">
                            </div>
                        </div>
                        <div id="uploadType1">
                            <div class="layui-form-item">
                                <label class="layui-form-label">文件:</label>
                                <div class="layui-input-block">
                                    <span class="gougu-upload-files">${typeTps[type]}</span><button type="button" class="layui-btn layui-btn-normal" onclick="document.querySelector('#chunkFile').click()">选择文件</button>
                                    <input type="file" id="chunkFile" class="layui-upload-file" accept="${typeExt[type]}">
                                </div>
                            </div>
                            <div class="layui-form-item">
                                <label class="layui-form-label"></label>
                                <div class="layui-input-block">
                                    <span class="gougu-upload-tips">只能上传 ${typeTps[type]} 文件</span>
                                </div>
                            </div>
                            <div class="layui-progress upload-progress" lay-showpercent="yes" lay-filter="upload-progress" style="margin-left:100px; width:420px; display:none;">
                                <div class="layui-progress-bar layui-bg-blue" lay-percent=""><span class="layui-progress-text"></span></div>
                                <div style="padding-top:15px; font-size:12px" id="gougu-upload-choosed"></div>    
                                <div style="padding-top:12px; display:none;">
                                    <button type="button" class="layui-btn layui-btn-sm layui-btn-danger" id="chunkDel">取消上传</button>
                                </div>
                            </div>
                        </div>
                        <div id="uploadType2" style="display:none; width:500px;">
                            <div class="layui-form-item">
                                <label class="layui-form-label">URL地址:</label>
                                <div class="layui-input-block">
                                    <input type="text" name="web_url" placeholder="" autocomplete="off" class="layui-input">
                                </div>
                            </div>
                            <div class="layui-form-item layui-form-item-sm">
                                <label class="layui-form-label"></label>
                                <div class="layui-input-block">
                                    <span class="layui-btn" id="saveAjax">确定提交</span>
                                </div>
                            </div>
                        </div>     
                </div>`,
                success: function(layero, index){
                    form.render();

                    form.on('radio(type)', function(data){
                        if(data.value==1){
                            $('#uploadType1').show();
                            $('#uploadType2').hide();
                        }
                        else{
                            $('#uploadType1').hide();
                            $('#uploadType2').show();
                        }
                    });  

                    // 监听文件上传
                    $('#chunkFile').change(function (e) {
                        var uploadFile = $(this)[0].files[0];
                        console.log(uploadFile,e);
                        var fileReader = new FileReader();
                        var fileId = getRandom();

                        $('#chunkDel').on('click',function(){
                            layer.confirm('确定要取消上传吗?', {icon: 3, title:'提示'}, function(idx){
                                loadding = false;
                                setTimeout(() => {
                                    clearChunk()
                                },500);
                                layer.close(idx);
                            });                
                        });

                        //读取文件名称
                        var fileName = uploadFile.name;
                        //读取文件后缀
                        var fileExtension = fileName.substring(fileName.lastIndexOf('.') + 1);
                        //读取文件大小
                        var fileSize = uploadFile.size;

                        // 定义每个区块的大小,以字节为单位
                        var chunk = 10 * 1024 * 1024;//每片10M
                        // 初始化记录的进度
                        var loaded = 0,loadding=true;
                        readPartFile(loaded,chunk);
                        // 监听加载完毕的区块
                        fileReader.onload = function (event) {
                            // 更新进度
                            loaded += chunk;
                            var targetResult = event.target.result;
                            // 转换成Blob对象
                            var targetBlob = new Blob([targetResult]);
                            console.log('event',event,'mime',targetBlob);
                            var progress = (loaded*100/fileSize).toFixed(0);
                            if(progress>=100){
                                progress = 99;
                            }

                            $('#gougu-upload-choosed').html('正在上传:'+fileName);
                            $('.upload-progress').show();
                            element.progress('upload-progress', progress + '%');
                            //$('#load').html(progress+"%");
                            var isEnd = loaded >= fileSize ? 1 : 0;
                            //每1秒上传一次,防止请求频率过快,时间可自定义
                            setTimeout(() => {
                                pieceUpload(targetBlob,isEnd,fileId,fileName,function (e) {
                                    console.log(e);
                                    if (isEnd) {
                                        loadding=false;
                                        layer.msg(e.msg);
                                        $('[name="file_id"]').val(e.data.id);
                                        $('[name="url"]').val(e.data.filepath);
                                        layer.close(index);
                                        return e;
                                    }
                                    //当前区块上传完毕后再上传下一个区块
                                    if(loadding == true){
                                        readPartFile(loaded,chunk);
                                    }
                                })
                            },500)

                        }
                        // 将文件分成指定大小的区块,start是起始位置
                        function readPartFile(start) {
                            var piece = uploadFile.slice(start,start + chunk);
                            fileReader.readAsArrayBuffer(piece);
                        }
                        // 分块上传 file:上传的区块 isEnd:是否已整体上传结束 fileId:源文件的唯一标识 filename:文件名 callback:上传之后的回调
                        function pieceUpload(file,isEnd,fileId,filename,callback) {
                            try {
                                var formData = new FormData();

                                formData.append('file',file,'file');
                                formData.append('is_end',isEnd);
                                formData.append('file_id',fileId);
                                formData.append('file_name',filename);
                                formData.append('file_extension',fileExtension);
                                formData.append('file_size',fileSize);
                                //向服务端表明该请求是一个分块上传
                                formData.append('type','chunk');
                                console.log(formData);
                                $.ajax({
                                //分块上传的接口url
                                    url: '/api/index/chunkUpload',
                                    type: 'POST',
                                    data: formData,
                                    processData: false,
                                    contentType: false,
                                    success: callback,
                                    error: function (err) {
                                        console.log(err)
                                    }
                                })
                            } catch (e) {
                                console.log(e)
                            }
                        }

                        function clearChunk(){
                            //删除分片上传的临时文件
                            $.ajax({
                                url: '/api/index/clearChunk',
                                type: 'POST',
                                data: {'file_id':fileId},
                                success: function(e){
                                    layer.msg(e.msg);
                                    layer.close(index);
                                },
                                error: function (err) {
                                    console.log(err);
                                }
                            })
                        }

                        //生成32位随机数
                        function getRandom(){
                            var chars = ['0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z'];
                            var str="";
                            for(var i=0;i<32;i++){
                                var id = parseInt(Math.random()*35);
                                str+=chars[id];
                            }
                            return str;
                        }
                    })

                    $('#saveAjax').on('click',function(){
                        let url=$('[name="web_url"]').val();
                        if(url == ''){
                            layer.msg('请输入网络URL');
                            return false;
                        }
                        $('[name="url"]').val(url);
                        layer.close(index);
                    })
                }
            });    
        }
<?php
declare (strict_types = 1);
namespace app\api\controller;

use app\api\BaseController;
use think\facade\Db;

class Index extends BaseController
{
//执行分片上传的控制器方法
public function chunkUpload() {
    if ($this->request->isPost()) {
        //执行分片上传流程
        $data = $this->request->post();
        //判断是否是分片上传
        if ($data['type'] === 'chunk') {
            $file = request()->file('file');
            //获取对应的上传配置
            $fs = \think\facade\Filesystem::disk('public');
            $ext = $file->extension();
            $chunkPath = $data['file_id'].'/'.$file->md5().($ext ? '.'.$ext : '');
            //存储分片文件到指定路径
            $savename = $fs->putFileAs( 'chunk', $file,$chunkPath);
            if (!$savename) {
                return json([
                    'code' => 1,
                    'msg' => '上传失败',
                    'data' => [],
                ]);
            }
            if (!$data['is_end']) {
                $filepath = '';
            } else {
                //合并块文件
                $fileUrl = '';
                $chunkSaveDir = \think\facade\Filesystem::getDiskConfig('public');
                $smallChunkDir = $chunkSaveDir['root'].'/chunk/'.$data['file_id'];
                //获取已存储的属于源文件的所有分块文件 进行合并
                if ($handle = opendir($smallChunkDir)) {
                    $chunkList = [];
                    $modifyTime = [];
                    while (false !== ($file = readdir($handle))) {
                        if ($file != "." && $file != "..") {
                            $temp['path'] = $smallChunkDir.'/'.$file;
                            $temp['modify'] = filemtime($smallChunkDir.'/'.$file);
                            $chunkList[] = $temp;
                            $modifyTime[] = $temp['modify'];
                        }
                    }
                    //对分块文件进行排序
                    array_multisort($modifyTime,SORT_ASC,$chunkList);
                    $saveDir = \think\facade\Filesystem::getDiskConfig('public');
                    $saveName = md5($data['file_id'].$data['file_name']).'.'.$data['file_extension'];
                    $newPath = $saveDir['root'].'/'.date('Ym').'/'.$saveName;
                    if (!file_exists($saveDir['root'].'/'.date('Ym'))) {
                        mkdir($saveDir['root'].'/'.date('Ym'),0777,true);
                    }
                    $newFileHandle = fopen($newPath,'a+b');
                    foreach ($chunkList as $item) {
                        fwrite($newFileHandle,file_get_contents($item['path']));
                        unlink($item['path']);
                    }
                    rmdir($smallChunkDir);
                    //将合并后的文件存储到指定路径
                    $fileUrl = $saveDir['url'].'/'.date('Ym').'/'.$saveName;
                    fclose($newFileHandle);
                    closedir($handle);
                } else {
                    return json([
                        'code' => 1,
                        'msg' => '目录:'.$chunkSaveDir['root'].'/chunk/'.$data['file_id'].'不存在',
                        'data' => [],
                    ]);
                }
                $filepath = $fileUrl;
            }
            $res=[];
            //合并流程结束
            if ($filepath!='') {
                $fileinfo = [];
                $fileinfo['filepath'] = $filepath;
                $fileinfo['name'] = $data['file_name'];
                $fileinfo['fileext'] = $data['file_extension'];
                $fileinfo['filesize'] = $data['file_size'];
                $fileinfo['filename'] = date('Ym').'/'.$saveName;
                $fileinfo['sha1'] = $data['file_id'];
                $fileinfo['md5'] = $data['file_id'];
                $fileinfo['module'] = \think\facade\App::initialize()->http->getName();
                $fileinfo['action'] = app('request')->action();
                $fileinfo['uploadip'] = app('request')->ip();
                $fileinfo['create_time'] = time();
                $fileinfo['user_id'] = get_login_admin('id') ? get_login_admin('id') : 0;
                if ($fileinfo['module'] = 'admin') {
                    //通过后台上传的文件直接审核通过
                    $fileinfo['status'] = 1;
                    $fileinfo['admin_id'] = $fileinfo['user_id'];
                    $fileinfo['audit_time'] = time();
                }
                $fileinfo['use'] = 'big';
                $res['id'] = Db::name('file')->insertGetId($fileinfo);
                $res['filepath'] = $fileinfo['filepath'];
                $res['name'] = $fileinfo['name'];
                $res['filename'] = $fileinfo['filename'];
                $res['filesize'] = $fileinfo['filesize'];
                $res['fileext'] = $fileinfo['fileext'];
                add_log('upload', $fileinfo['user_id'], $fileinfo);
            } 
            return to_assign(0, '上传成功', $res);
        }
    }
}

//取消上传,删除临时文件
public function clearChunk() {
    if ($this->request->isPost()) {
        $param = get_params();
        $saveDir = \think\facade\Filesystem::getDiskConfig('public');
        $smallChunkDir = $saveDir['root'].'/chunk/'.$param['file_id'];
        if(!is_dir($smallChunkDir)){
            return to_assign(0, '上传的临时文件已删除');
        }
        //获取已存储的属于源文件的所有分块文件
        if ($handle = opendir($smallChunkDir)) {
            while (false !== ($file = readdir($handle))) {
                if ($file != "." && $file != "..") {
                    $temp['path'] = $smallChunkDir.'/'.$file;
                    unlink($temp['path']);
                }
            }
            rmdir($smallChunkDir);
            closedir($handle);
            return to_assign(0, '已取消上传');
        }
    }
}

PS: Nginx上传大文件超时的解决办法。
用nginx作代理服务器,上传大文件时(本人测试上传50m的文件),提示上传超时或文件过大。原因是nginx对上传文件大小有限制,而且默认是1M。另外,若上传文件很大,还要适当调整上传超时时间。
解决方法是在nginx的配置文件下,加上以下配置:

client_max_body_size     50m; //文件大小限制,默认1m
client_header_timeout    1m; 
client_body_timeout      1m; 
proxy_connect_timeout     60s; 
proxy_read_timeout      1m; 
proxy_send_timeout      1m;

PS:PHP上传大文件超过40秒服务器500的解决办法

找当前PHP版本的配置文件php.ini修改以下文件上传限制参数:
max_execution_time = 600
max_input_time = 600
memory_limit = 1024M
post_max_size = 1024M
upload_max_filesize = 1024M